[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:46 dist-test-slave-44d2 krb5kdc[1542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:46 dist-test-slave-44d2 krb5kdc[1542](info): set up 2 sockets
Aug 10 07:19:46 dist-test-slave-44d2 krb5kdc[1542](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 07:19:47.884850 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.378s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:19:47 dist-test-slave-44d2 krb5kdc[1542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:19:48.019743 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:46639
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:46639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:48.136510 1562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:48.136837 1562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:48.136917 1562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:48.140900 1562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:48.140997 1562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:48.141034 1562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:48.141073 1562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:48.141108 1562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:48.146370 1562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:46639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:46639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:48.147467 1562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:48.148547 1562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:48.158363 1569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.158363 1575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.158454 1572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:48.158969 1562 server_base.cc:1034] running on GCE node
I20260810 07:19:48.159659 1562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:48.160952 1562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:48.162184 1562 hybrid_clock.cc:648] HybridClock initialized: now 1786346388162147 us; error 81 us; skew 500 ppm
Aug 10 07:19:48 dist-test-slave-44d2 krb5kdc[1542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346388, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:48.165566 1562 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:19:48.166597 1562 webserver.cc:460] Webserver started at http://127.1.125.254:33887/ using document root <none> and password file <none>
I20260810 07:19:48.167271 1562 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:48.167378 1562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:48.167655 1562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:48.170107 1562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "2e3b728f4c994342b43f7b81008b2098"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-44d2"
server_key: "08487f58f2381af4a9f6c2c6e634dd03"
server_key_iv: "84bbee9807be6ff459030b8bb41f2a2e"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.170815 1562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "2e3b728f4c994342b43f7b81008b2098"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-44d2"
server_key: "08487f58f2381af4a9f6c2c6e634dd03"
server_key_iv: "84bbee9807be6ff459030b8bb41f2a2e"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.175552 1562 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:19:48.178345 1583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.179646 1562 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:19:48.179903 1562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "2e3b728f4c994342b43f7b81008b2098"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-44d2"
server_key: "08487f58f2381af4a9f6c2c6e634dd03"
server_key_iv: "84bbee9807be6ff459030b8bb41f2a2e"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.180051 1562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:48.209197 1562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:48.212765 1562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:48.213132 1562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:48.222950 1562 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:46639
I20260810 07:19:48.222959 1651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:46639 every 8 connection(s)
I20260810 07:19:48.224036 1562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:48.226795 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 1562
I20260810 07:19:48.226929 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:19:48.227119 1652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:48.228370 1527 external_mini_cluster.cc:1428] Setting key 22625572d81230de83dce8eccc1ef729
I20260810 07:19:48.233843 1652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098: Bootstrap starting.
I20260810 07:19:48.236267 1652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:48.237100 1652 log.cc:824] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:48.239550 1652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098: No bootstrap required, opened a new log
I20260810 07:19:48.244287 1652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e3b728f4c994342b43f7b81008b2098" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46639 } }
I20260810 07:19:48.244654 1652 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:48.244721 1652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e3b728f4c994342b43f7b81008b2098, State: Initialized, Role: FOLLOWER
I20260810 07:19:48.245247 1652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e3b728f4c994342b43f7b81008b2098" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46639 } }
I20260810 07:19:48.245445 1652 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:48.245532 1652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:48.245644 1652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:48.247112 1652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e3b728f4c994342b43f7b81008b2098" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46639 } }
I20260810 07:19:48.247539 1652 leader_election.cc:302] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e3b728f4c994342b43f7b81008b2098; no voters:
I20260810 07:19:48.248026 1652 leader_election.cc:288] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:48.248312 1660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:48.248601 1660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [term 1 LEADER]: Becoming Leader. State: Replica: 2e3b728f4c994342b43f7b81008b2098, State: Running, Role: LEADER
Aug 10 07:19:48 dist-test-slave-44d2 krb5kdc[1542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:48.248975 1660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e3b728f4c994342b43f7b81008b2098" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46639 } }
I20260810 07:19:48.249452 1652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:48.251909 1661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e3b728f4c994342b43f7b81008b2098" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e3b728f4c994342b43f7b81008b2098" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46639 } } }
I20260810 07:19:48.252069 1661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:48.252452 1662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e3b728f4c994342b43f7b81008b2098. Latest consensus state: current_term: 1 leader_uuid: "2e3b728f4c994342b43f7b81008b2098" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e3b728f4c994342b43f7b81008b2098" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46639 } } }
I20260810 07:19:48.252640 1662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:48.257145 1655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.231775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56554 (local address 127.1.125.254:46639)
0810 07:19:48.232202 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.232217 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.233359 (+ 1142us) server_negotiation.cc:407] Connection header received
0810 07:19:48.236746 (+ 3387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.236763 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.237076 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.237377 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.240390 (+ 3013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.241667 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.243663 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.244187 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.244331 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:48.250155 (+ 5824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.250164 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.250193 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:48.250233 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.253126 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:48.253872 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:48.253878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:48.253884 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:48.254023 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:48.254392 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:48.254396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:48.254397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:48.254875 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.255135 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.255707 (+ 572us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.256001 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":145,"threads_started":1}
W20260810 07:19:48.257370 1682 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:48.257460 1682 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:48.257550 1674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:48.258744 1674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:48.265177 1674 catalog_manager.cc:1347] Generated new cluster ID: 7756e0f669264e95af9ff489ed48d338
I20260810 07:19:48.265256 1674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:48.271440 1674 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:48.272305 1674 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:48.279794 1674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e3b728f4c994342b43f7b81008b2098: Generated new TSK 0
I20260810 07:19:48.280652 1674 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:19:48.316431 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46639
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:48.436754 1691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:48.437052 1691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:48.437134 1691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:48.441118 1691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:48.441210 1691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:48.441301 1691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:19:48.446534 1691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46639
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:48.447620 1691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:48.448879 1691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:48.460032 1700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.460026 1697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.460034 1703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:48.460521 1691 server_base.cc:1034] running on GCE node
I20260810 07:19:48.461099 1691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:48.461890 1691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:48.463094 1691 hybrid_clock.cc:648] HybridClock initialized: now 1786346388463075 us; error 39 us; skew 500 ppm
Aug 10 07:19:48 dist-test-slave-44d2 krb5kdc[1542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346388, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:48.466343 1691 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:19:48.467362 1691 webserver.cc:460] Webserver started at http://127.1.125.193:33343/ using document root <none> and password file <none>
I20260810 07:19:48.467962 1691 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:48.468060 1691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:48.468305 1691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:48.470585 1691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "81992dc5b0d647a8963cf13df54e6d26"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-44d2"
server_key: "b09fedda5ed995fc4caf83a8afc257d5"
server_key_iv: "feff1e27dc334c8e340fec5e44b83c01"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.471203 1691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "81992dc5b0d647a8963cf13df54e6d26"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-44d2"
server_key: "b09fedda5ed995fc4caf83a8afc257d5"
server_key_iv: "feff1e27dc334c8e340fec5e44b83c01"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.475476 1691 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260810 07:19:48.478211 1717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.479491 1691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:19:48.479635 1691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "81992dc5b0d647a8963cf13df54e6d26"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-44d2"
server_key: "b09fedda5ed995fc4caf83a8afc257d5"
server_key_iv: "feff1e27dc334c8e340fec5e44b83c01"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.479748 1691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:48.501034 1691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:48.504210 1691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:48.504428 1691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:48.505162 1691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:48.506095 1691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:48.506170 1691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.506246 1691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:48.506299 1691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.520004 1691 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:42781
I20260810 07:19:48.520040 1876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:42781 every 8 connection(s)
I20260810 07:19:48.521061 1691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:48.524032 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 1691
I20260810 07:19:48.524149 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:48.524456 1527 external_mini_cluster.cc:1428] Setting key 9ab5c7f074f3bfd66685a98285e87dff
Aug 10 07:19:48 dist-test-slave-44d2 krb5kdc[1542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346388, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:19:48.537894 1655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.523147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:58457 (local address 127.1.125.254:46639)
0810 07:19:48.523291 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.523294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.524249 (+ 955us) server_negotiation.cc:407] Connection header received
0810 07:19:48.526274 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.526277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.526321 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.526395 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.528149 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.528918 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.530327 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.530775 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.530909 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:48.533958 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.533961 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.533963 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:48.533989 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.535557 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:48.536207 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:48.536209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:48.536210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:48.536286 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:48.536778 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:48.536780 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:48.536782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:48.536983 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.537085 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.537649 (+ 564us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.537763 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:19:48.538784 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.523509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:46639 (local address 127.1.125.193:58457)
0810 07:19:48.524062 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:48.524143 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:48.525993 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:48.526611 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:48.526624 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:48.527208 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:48.527960 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:48.527984 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.529056 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.529059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:48.530178 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:48.530191 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.530951 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.530954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:48.531111 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:48.531762 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:48.531779 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:48.533771 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:48.535686 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:48.535693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:48.535710 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:48.536104 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:48.536391 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:48.536396 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:48.536400 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:48.536670 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:48.537138 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:48.537143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:48.537382 (+ 239us) client_negotiation.cc:769] Sending connection context
0810 07:19:48.537553 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 07:19:48.537821 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":115,"threads_started":1}
I20260810 07:19:48.540187 1878 heartbeater.cc:347] Connected to a master server at 127.1.125.254:46639
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
I20260810 07:19:48.540460 1878 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:48.541038 1878 heartbeater.cc:511] Master 127.1.125.254:46639 requested a full tablet report, sending...
I20260810 07:19:48.542800 1606 ts_manager.cc:194] Registered new tserver with Master: 81992dc5b0d647a8963cf13df54e6d26 (127.1.125.193:42781)
I20260810 07:19:48.544011 1606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:58457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:19:48.559453 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46639
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:48.683785 1886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:48.684105 1886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:48.684185 1886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:48.688472 1886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:48.688547 1886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:48.688647 1886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:19:48.694022 1886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46639
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:48.695098 1886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:48.696146 1886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:48.706507 1896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.706539 1893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.706569 1900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:48.706954 1886 server_base.cc:1034] running on GCE node
I20260810 07:19:48.707327 1886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:48.707964 1886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:48.709178 1886 hybrid_clock.cc:648] HybridClock initialized: now 1786346388709157 us; error 36 us; skew 500 ppm
Aug 10 07:19:48 dist-test-slave-44d2 krb5kdc[1542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346388, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:48.712181 1886 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:19:48.713444 1886 webserver.cc:460] Webserver started at http://127.1.125.194:41127/ using document root <none> and password file <none>
I20260810 07:19:48.714056 1886 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:48.714164 1886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:48.714429 1886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:48.716907 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "d3e74a2f6efe4b1f9883e2d005e1dc01"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-44d2"
server_key: "9bc5d679f89c6be491806a997a3b08e6"
server_key_iv: "a6b4d896e4c80ce14a005c1cdfcfe358"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.717535 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "d3e74a2f6efe4b1f9883e2d005e1dc01"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-44d2"
server_key: "9bc5d679f89c6be491806a997a3b08e6"
server_key_iv: "a6b4d896e4c80ce14a005c1cdfcfe358"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.721624 1886 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:19:48.724334 1913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.725644 1886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:48.725795 1886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "d3e74a2f6efe4b1f9883e2d005e1dc01"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-44d2"
server_key: "9bc5d679f89c6be491806a997a3b08e6"
server_key_iv: "a6b4d896e4c80ce14a005c1cdfcfe358"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.725919 1886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:48.751734 1886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:48.755383 1886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:48.755623 1886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:48.756295 1886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:48.757570 1886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:48.757653 1886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.757733 1886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:48.757791 1886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.771129 1886 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:42069
I20260810 07:19:48.771149 2068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:42069 every 8 connection(s)
I20260810 07:19:48.772192 1886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:48.777009 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 1886
I20260810 07:19:48.777155 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:48.777462 1527 external_mini_cluster.cc:1428] Setting key b1effc53d2b641cebbaa40b3501122cc
Aug 10 07:19:48 dist-test-slave-44d2 krb5kdc[1542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346388, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:48.788261 1655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.774453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:45131 (local address 127.1.125.254:46639)
0810 07:19:48.774617 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:48.774622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:48.775589 (+ 967us) server_negotiation.cc:407] Connection header received
0810 07:19:48.777270 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:48.777274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:48.777337 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:48.777444 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:48.778687 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.779343 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.780763 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.781004 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.781123 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:48.784278 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:48.784282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:48.784284 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:48.784310 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:48.786065 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:48.786610 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:48.786615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:48.786617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:48.786702 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:48.787076 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:48.787080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:48.787082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:48.787268 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:48.787464 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:48.787964 (+ 500us) server_negotiation.cc:299] Negotiation successful
0810 07:19:48.788093 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
I20260810 07:19:48.789261 2071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:48.774831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:46639 (local address 127.1.125.194:45131)
0810 07:19:48.775422 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:48.775467 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:48.776984 (+ 1517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:48.777635 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:48.777643 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:48.778014 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:48.778530 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:48.778544 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.779486 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.779489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:48.780631 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:48.780643 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:48.781170 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:48.781174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:48.781397 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:48.782195 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:48.782222 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:48.784102 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:48.786193 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:48.786199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:48.786216 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:48.786501 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:48.786812 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:48.786816 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:48.786818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:48.786974 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:48.787438 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:48.787447 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:48.787711 (+ 264us) client_negotiation.cc:769] Sending connection context
0810 07:19:48.787917 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 07:19:48.788258 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":159,"threads_started":1}
I20260810 07:19:48.790684 2069 heartbeater.cc:347] Connected to a master server at 127.1.125.254:46639
I20260810 07:19:48.790984 2069 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:48.791577 2069 heartbeater.cc:511] Master 127.1.125.254:46639 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
I20260810 07:19:48.792975 1606 ts_manager.cc:194] Registered new tserver with Master: d3e74a2f6efe4b1f9883e2d005e1dc01 (127.1.125.194:42069)
I20260810 07:19:48.793668 1606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:45131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:19:48.811702 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46639
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:48.936414 2076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:48.936766 2076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:48.936815 2076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:48.940734 2076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:48.940788 2076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:48.940860 2076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:19:48.946048 2076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46639
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:48.947093 2076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:48.948096 2076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:48.959110 2083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.959172 2086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:48.959118 2084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:48.959525 2076 server_base.cc:1034] running on GCE node
I20260810 07:19:48.959969 2076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:48.960692 2076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:48.961884 2076 hybrid_clock.cc:648] HybridClock initialized: now 1786346388961848 us; error 49 us; skew 500 ppm
Aug 10 07:19:48 dist-test-slave-44d2 krb5kdc[1542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346388, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:48.964838 2076 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:19:48.965798 2076 webserver.cc:460] Webserver started at http://127.1.125.195:45429/ using document root <none> and password file <none>
I20260810 07:19:48.966395 2076 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:48.966493 2076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:48.966746 2076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:48.968892 2076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "19209e9c339f4d50b57c876ff6bf3f18"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-44d2"
server_key: "8aeed4441b85a73283c6e48b9131658e"
server_key_iv: "7b57fb0c749277205b990c7a25b2c047"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.969566 2076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "19209e9c339f4d50b57c876ff6bf3f18"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-44d2"
server_key: "8aeed4441b85a73283c6e48b9131658e"
server_key_iv: "7b57fb0c749277205b990c7a25b2c047"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.973342 2076 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 07:19:48.975723 2097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.976984 2076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:19:48.977126 2076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "19209e9c339f4d50b57c876ff6bf3f18"
format_stamp: "Formatted at 2026-08-10 07:19:48 on dist-test-slave-44d2"
server_key: "8aeed4441b85a73283c6e48b9131658e"
server_key_iv: "7b57fb0c749277205b990c7a25b2c047"
server_key_version: "encryptionkey@0"
I20260810 07:19:48.977238 2076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:48.991091 2076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:48.994206 2076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:48.994546 2076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:48.995186 2076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:48.996069 2076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:48.996168 2076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:48.996245 2076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:48.996302 2076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:49.008009 2076 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:45121
I20260810 07:19:49.008033 2278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:45121 every 8 connection(s)
I20260810 07:19:49.008999 2076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:49.010030 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 2076
I20260810 07:19:49.010157 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:49.010445 1527 external_mini_cluster.cc:1428] Setting key a0c4fe6e31af8d18a9eccea1bb1b4fa4
Aug 10 07:19:49 dist-test-slave-44d2 krb5kdc[1542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346388, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:49.022838 1655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:49.011170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:50607 (local address 127.1.125.254:46639)
0810 07:19:49.011346 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:49.011350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:49.012328 (+ 978us) server_negotiation.cc:407] Connection header received
0810 07:19:49.013794 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:49.013797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:49.013842 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:49.013911 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:49.015050 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.015562 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.016765 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.016991 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.017065 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:49.019580 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:49.019584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:49.019587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:49.019612 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:49.020965 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:49.021436 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:49.021439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:49.021440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:49.021514 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:49.021815 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:49.021817 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:49.021818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:49.021988 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:49.022065 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:49.022605 (+ 540us) server_negotiation.cc:299] Negotiation successful
0810 07:19:49.022713 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:19:49.023730 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:49.011682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:46639 (local address 127.1.125.195:50607)
0810 07:19:49.012170 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:49.012228 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:49.013602 (+ 1374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:49.014044 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:49.014051 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:49.014407 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:49.014909 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:49.014922 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.015673 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.015675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:49.016657 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:49.016668 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.017119 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.017122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:49.017260 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:49.017926 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:49.017943 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:49.019417 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:49.021080 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:49.021085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:49.021097 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:49.021349 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:49.021602 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:49.021604 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:49.021605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:49.021734 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:49.022077 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:49.022082 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:49.022394 (+ 312us) client_negotiation.cc:769] Sending connection context
0810 07:19:49.022554 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 07:19:49.022800 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":144,"threads_started":1}
I20260810 07:19:49.025010 2280 heartbeater.cc:347] Connected to a master server at 127.1.125.254:46639
I20260810 07:19:49.025241 2280 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:49.025743 2280 heartbeater.cc:511] Master 127.1.125.254:46639 requested a full tablet report, sending...
I20260810 07:19:49.026741 1606 ts_manager.cc:194] Registered new tserver with Master: 19209e9c339f4d50b57c876ff6bf3f18 (127.1.125.195:45121)
I20260810 07:19:49.027295 1606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:50607
I20260810 07:19:49.036854 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:19:49 dist-test-slave-44d2 krb5kdc[1542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346389, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:19:49 dist-test-slave-44d2 krb5kdc[1542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346389, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.125.193@KRBTEST.COM
I20260810 07:19:49.066216 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:49.056124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41966 (local address 127.1.125.193:42781)
0810 07:19:49.056465 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:49.056474 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:49.056525 (+ 51us) server_negotiation.cc:407] Connection header received
0810 07:19:49.056706 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:49.056714 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:49.056920 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:49.057095 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:49.057971 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.058740 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.059604 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.059989 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.060123 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:49.062508 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:49.062525 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:49.062539 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:49.062563 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:49.064235 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:49.064770 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:49.064776 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:49.064777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:49.064870 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:49.065176 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:49.065179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:49.065182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:49.065440 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:49.065669 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:49.065874 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 07:19:49.066041 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":95,"threads_started":1}
W20260810 07:19:49.067245 1757 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41966
Aug 10 07:19:49 dist-test-slave-44d2 krb5kdc[1542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:19:49 dist-test-slave-44d2 krb5kdc[1542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.193@KRBTEST.COM
I20260810 07:19:49.096026 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:49.087580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41980 (local address 127.1.125.193:42781)
0810 07:19:49.087756 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:49.087760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:49.087811 (+ 51us) server_negotiation.cc:407] Connection header received
0810 07:19:49.088036 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:49.088041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:49.088100 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:49.088174 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:49.088969 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.089499 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.090304 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:49.090545 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:49.090622 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:49.093135 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:49.093139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:49.093141 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:49.093168 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:49.094500 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:49.094879 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:49.094882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:49.094883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:49.094959 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:49.095216 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:49.095218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:49.095219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:49.095374 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:49.095472 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:49.095674 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 07:19:49.095775 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 07:19:49.097230 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 1691
I20260810 07:19:49.110184 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 1886
I20260810 07:19:49.127274 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 2076
I20260810 07:19:49.146312 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 1562
2026-08-10T07:19:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2679 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:49 dist-test-slave-44d2 krb5kdc[2300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:49 dist-test-slave-44d2 krb5kdc[2300](info): set up 2 sockets
Aug 10 07:19:49 dist-test-slave-44d2 krb5kdc[2300](info): commencing operation
krb5kdc: starting...
W20260810 07:19:50.451614 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:19:50 dist-test-slave-44d2 krb5kdc[2300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:19:50.583462 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:39825
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:39825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:50.702934 2320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:50.703267 2320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:50.703363 2320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:50.707389 2320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:50.707484 2320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:50.707530 2320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:50.707590 2320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:50.707633 2320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:50.712996 2320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:39825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:39825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:50.714131 2320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:50.715201 2320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:50.724161 2331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:50.724269 2320 server_base.cc:1034] running on GCE node
W20260810 07:19:50.724170 2327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:50.724217 2333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:50.725008 2320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:50.726172 2320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:50.727358 2320 hybrid_clock.cc:648] HybridClock initialized: now 1786346390727327 us; error 41 us; skew 500 ppm
Aug 10 07:19:50 dist-test-slave-44d2 krb5kdc[2300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346390, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:50.730214 2320 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:19:50.731106 2320 webserver.cc:460] Webserver started at http://127.1.125.254:34661/ using document root <none> and password file <none>
I20260810 07:19:50.731727 2320 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:50.731792 2320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:50.732043 2320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:50.734206 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "9a246a51dd5c42a1a701868c27d8d545"
format_stamp: "Formatted at 2026-08-10 07:19:50 on dist-test-slave-44d2"
server_key: "fee7ad6595d8f33cb610fc10db448189"
server_key_iv: "5d36da49760a3809ef2fb57925bc6d8a"
server_key_version: "encryptionkey@0"
I20260810 07:19:50.734791 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "9a246a51dd5c42a1a701868c27d8d545"
format_stamp: "Formatted at 2026-08-10 07:19:50 on dist-test-slave-44d2"
server_key: "fee7ad6595d8f33cb610fc10db448189"
server_key_iv: "5d36da49760a3809ef2fb57925bc6d8a"
server_key_version: "encryptionkey@0"
I20260810 07:19:50.738667 2320 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260810 07:19:50.741156 2346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:50.742369 2320 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:50.742506 2320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "9a246a51dd5c42a1a701868c27d8d545"
format_stamp: "Formatted at 2026-08-10 07:19:50 on dist-test-slave-44d2"
server_key: "fee7ad6595d8f33cb610fc10db448189"
server_key_iv: "5d36da49760a3809ef2fb57925bc6d8a"
server_key_version: "encryptionkey@0"
I20260810 07:19:50.742621 2320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:50.767221 2320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:50.770628 2320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:50.770859 2320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:50.778970 2320 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:39825
I20260810 07:19:50.778980 2426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:39825 every 8 connection(s)
I20260810 07:19:50.780035 2320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:50.783159 2429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:50.788789 2429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545: Bootstrap starting.
I20260810 07:19:50.789913 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 2320
I20260810 07:19:50.790052 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:19:50.790364 1527 external_mini_cluster.cc:1428] Setting key d4cd874fbff2d9169c3ad63af16eaba3
I20260810 07:19:50.791883 2429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:50.792801 2429 log.cc:824] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:50.794697 2429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545: No bootstrap required, opened a new log
I20260810 07:19:50.798130 2429 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a246a51dd5c42a1a701868c27d8d545" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39825 } }
I20260810 07:19:50.798377 2429 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:50.798461 2429 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a246a51dd5c42a1a701868c27d8d545, State: Initialized, Role: FOLLOWER
I20260810 07:19:50.798947 2429 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a246a51dd5c42a1a701868c27d8d545" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39825 } }
I20260810 07:19:50.799101 2429 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:50.799190 2429 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:50.799300 2429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [term 0 FOLLOWER]: Advancing to term 1
Aug 10 07:19:50 dist-test-slave-44d2 krb5kdc[2300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:50.800462 2429 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a246a51dd5c42a1a701868c27d8d545" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39825 } }
I20260810 07:19:50.800882 2429 leader_election.cc:302] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a246a51dd5c42a1a701868c27d8d545; no voters:
I20260810 07:19:50.801308 2429 leader_election.cc:288] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:50.801424 2434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:50.801643 2434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [term 1 LEADER]: Becoming Leader. State: Replica: 9a246a51dd5c42a1a701868c27d8d545, State: Running, Role: LEADER
I20260810 07:19:50.801952 2434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a246a51dd5c42a1a701868c27d8d545" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39825 } }
I20260810 07:19:50.802696 2429 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:50.803583 2435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a246a51dd5c42a1a701868c27d8d545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a246a51dd5c42a1a701868c27d8d545" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39825 } } }
I20260810 07:19:50.803743 2435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:50.804164 2447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:50.804111 2436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a246a51dd5c42a1a701868c27d8d545. Latest consensus state: current_term: 1 leader_uuid: "9a246a51dd5c42a1a701868c27d8d545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a246a51dd5c42a1a701868c27d8d545" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39825 } } }
I20260810 07:19:50.804207 2436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:50.806385 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:50.791570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54888 (local address 127.1.125.254:39825)
0810 07:19:50.792142 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:50.792158 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:50.792195 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:19:50.792951 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:50.792982 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:50.793327 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:50.793672 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:50.794515 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:50.795564 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:50.796440 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:50.796989 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:50.797221 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:50.800125 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:50.800132 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:50.800146 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:50.800174 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:50.802307 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:50.802853 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:50.802859 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:50.802865 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:50.803014 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:50.803399 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:50.803404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:50.803407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:50.803993 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:50.804865 (+ 872us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:50.805066 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 07:19:50.805415 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":135,"threads_started":1}
I20260810 07:19:50.807354 2447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:50.813280 2447 catalog_manager.cc:1347] Generated new cluster ID: ba50dcd4c958421294ef8997f5071845
I20260810 07:19:50.813354 2447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:50.822860 2447 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:50.823825 2447 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:50.832989 2447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a246a51dd5c42a1a701868c27d8d545: Generated new TSK 0
I20260810 07:19:50.833753 2447 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:19:50.881088 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39825
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:50.999094 2470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:50.999403 2470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:50.999490 2470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:51.003695 2470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:51.003779 2470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:51.003870 2470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:19:51.009441 2470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39825
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:51.010535 2470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:51.011593 2470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:51.021661 2483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:51.021661 2479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:51.021909 2470 server_base.cc:1034] running on GCE node
W20260810 07:19:51.021721 2481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:51.022533 2470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:51.023207 2470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:51.024384 2470 hybrid_clock.cc:648] HybridClock initialized: now 1786346391024361 us; error 31 us; skew 500 ppm
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[2300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.027371 2470 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:19:51.028328 2470 webserver.cc:460] Webserver started at http://127.1.125.193:39329/ using document root <none> and password file <none>
I20260810 07:19:51.029004 2470 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:51.029104 2470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:51.029356 2470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:51.031527 2470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "0fa33829b5fa4fc59863fe587d8aeff6"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-44d2"
server_key: "13af1a01272c448e29c21717c4356ebe"
server_key_iv: "faf3662b8a502744b64e139380da42b9"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.032128 2470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "0fa33829b5fa4fc59863fe587d8aeff6"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-44d2"
server_key: "13af1a01272c448e29c21717c4356ebe"
server_key_iv: "faf3662b8a502744b64e139380da42b9"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.035975 2470 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:19:51.038630 2492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.039980 2470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:19:51.040128 2470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "0fa33829b5fa4fc59863fe587d8aeff6"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-44d2"
server_key: "13af1a01272c448e29c21717c4356ebe"
server_key_iv: "faf3662b8a502744b64e139380da42b9"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.040244 2470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:51.058840 2470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:51.062340 2470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:51.062566 2470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:51.063210 2470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:51.064092 2470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:51.064170 2470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.064244 2470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:51.064291 2470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.074208 2470 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:44405
I20260810 07:19:51.074231 2660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:44405 every 8 connection(s)
I20260810 07:19:51.075129 2470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:51.079450 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 2470
I20260810 07:19:51.079566 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:51.079849 1527 external_mini_cluster.cc:1428] Setting key 3985302b0d066ea403e83d3dee1f4494
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[2300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:19:51.091964 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.077306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:56281 (local address 127.1.125.254:39825)
0810 07:19:51.077485 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.077489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.078419 (+ 930us) server_negotiation.cc:407] Connection header received
0810 07:19:51.080199 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.080202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.080288 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.080398 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.082087 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.082618 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.084174 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.084623 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.084748 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.087672 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.087679 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.087686 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.087723 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.089675 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.090294 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.090300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.090305 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.090409 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.090804 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.090811 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.090815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.091060 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.091167 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.091718 (+ 551us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.091833 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 07:19:51.093094 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.077605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39825 (local address 127.1.125.193:56281)
0810 07:19:51.078231 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:51.078275 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:51.079945 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:51.080650 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:51.080659 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:51.081213 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:51.081918 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.081943 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.082755 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.082762 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.084046 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.084056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.084774 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.084777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.084978 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.085631 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:51.085648 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:51.087476 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:51.089814 (+ 2338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:51.089818 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:51.089831 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:51.090176 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:51.090544 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:51.090547 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:51.090549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:51.090688 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:51.091171 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:51.091176 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:51.091469 (+ 293us) client_negotiation.cc:769] Sending connection context
0810 07:19:51.091635 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 07:19:51.091889 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":126,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
I20260810 07:19:51.094471 2662 heartbeater.cc:347] Connected to a master server at 127.1.125.254:39825
I20260810 07:19:51.094764 2662 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:51.095402 2662 heartbeater.cc:511] Master 127.1.125.254:39825 requested a full tablet report, sending...
I20260810 07:19:51.097353 2371 ts_manager.cc:194] Registered new tserver with Master: 0fa33829b5fa4fc59863fe587d8aeff6 (127.1.125.193:44405)
I20260810 07:19:51.098603 2371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:56281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:19:51.111477 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39825
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:51.235466 2670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:51.235728 2670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:51.235805 2670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:51.240334 2670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:51.240409 2670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:51.240543 2670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:19:51.246121 2670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39825
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:51.247267 2670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:51.248435 2670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:51.258780 2687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:51.258810 2684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:51.258822 2685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:51.258958 2670 server_base.cc:1034] running on GCE node
I20260810 07:19:51.259510 2670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:51.260198 2670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:51.261439 2670 hybrid_clock.cc:648] HybridClock initialized: now 1786346391261417 us; error 45 us; skew 500 ppm
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[2300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.264638 2670 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:19:51.265511 2670 webserver.cc:460] Webserver started at http://127.1.125.194:44171/ using document root <none> and password file <none>
I20260810 07:19:51.266108 2670 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:51.266173 2670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:51.266345 2670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:51.268467 2670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "31e006e3262948e399f3df0d9b21154c"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-44d2"
server_key: "267f570dd7bcf3a79fb66c38b279b867"
server_key_iv: "388ac3d24110828930bdb853ee267c83"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.269112 2670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "31e006e3262948e399f3df0d9b21154c"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-44d2"
server_key: "267f570dd7bcf3a79fb66c38b279b867"
server_key_iv: "388ac3d24110828930bdb853ee267c83"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.273085 2670 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 07:19:51.275626 2696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.277050 2670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:51.277189 2670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "31e006e3262948e399f3df0d9b21154c"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-44d2"
server_key: "267f570dd7bcf3a79fb66c38b279b867"
server_key_iv: "388ac3d24110828930bdb853ee267c83"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.277298 2670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:51.298889 2670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:51.302409 2670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:51.302592 2670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:51.303464 2670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:51.304661 2670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:51.304716 2670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.305138 2670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:51.305174 2670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.314357 2670 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:35055
I20260810 07:19:51.314385 2870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:35055 every 8 connection(s)
I20260810 07:19:51.315259 2670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:51.319200 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 2670
I20260810 07:19:51.319291 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:51.319552 1527 external_mini_cluster.cc:1428] Setting key 0c557d27fd96d98db59c46129853924d
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[2300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:51.329988 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.317166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:55067 (local address 127.1.125.254:39825)
0810 07:19:51.317304 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.317307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.318117 (+ 810us) server_negotiation.cc:407] Connection header received
0810 07:19:51.319566 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.319569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.319630 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.319717 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.321202 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.321743 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.322928 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.323170 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.323282 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.326123 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.326127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.326130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.326166 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.327659 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.328239 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.328242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.328244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.328325 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.328744 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.328748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.328750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.328991 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.329112 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.329669 (+ 557us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.329800 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
I20260810 07:19:51.330880 2874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.317497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39825 (local address 127.1.125.194:55067)
0810 07:19:51.317951 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:51.317996 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:51.319321 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:51.319894 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:51.319901 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:51.320327 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:51.321041 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.321065 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.321877 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.321881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.322788 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.322799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.323337 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.323341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.323522 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.324277 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:51.324294 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:51.325879 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:51.327790 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:51.327796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:51.327810 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:51.328096 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:51.328423 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:51.328427 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:51.328429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:51.328640 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:51.329107 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:51.329113 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:51.329405 (+ 292us) client_negotiation.cc:769] Sending connection context
0810 07:19:51.329611 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 07:19:51.329952 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":110,"threads_started":1}
I20260810 07:19:51.332054 2871 heartbeater.cc:347] Connected to a master server at 127.1.125.254:39825
I20260810 07:19:51.332321 2871 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:51.332974 2871 heartbeater.cc:511] Master 127.1.125.254:39825 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
I20260810 07:19:51.334211 2371 ts_manager.cc:194] Registered new tserver with Master: 31e006e3262948e399f3df0d9b21154c (127.1.125.194:35055)
I20260810 07:19:51.334854 2371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:55067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:19:51.352353 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39825
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:51.469197 2877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:51.469449 2877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:51.469501 2877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:51.473464 2877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:51.473524 2877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:51.473582 2877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:19:51.478736 2877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39825
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:51.479712 2877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:51.480798 2877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:51.490724 2895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:51.490700 2892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:51.490752 2877 server_base.cc:1034] running on GCE node
W20260810 07:19:51.490681 2889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:51.491307 2877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:51.491935 2877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:51.493098 2877 hybrid_clock.cc:648] HybridClock initialized: now 1786346391493081 us; error 27 us; skew 500 ppm
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[2300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:51.495903 2877 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:19:51.496825 2877 webserver.cc:460] Webserver started at http://127.1.125.195:38161/ using document root <none> and password file <none>
I20260810 07:19:51.497437 2877 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:51.497504 2877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:51.497685 2877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:51.499691 2877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "7b7117c50b274d27b86a94dc730b2208"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-44d2"
server_key: "e60725dad8d936eba31d297787726444"
server_key_iv: "3df7fd318e24972827c2334815482fb8"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.500227 2877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "7b7117c50b274d27b86a94dc730b2208"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-44d2"
server_key: "e60725dad8d936eba31d297787726444"
server_key_iv: "3df7fd318e24972827c2334815482fb8"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.504465 2877 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 07:19:51.507006 2908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.508224 2877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:19:51.508381 2877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "7b7117c50b274d27b86a94dc730b2208"
format_stamp: "Formatted at 2026-08-10 07:19:51 on dist-test-slave-44d2"
server_key: "e60725dad8d936eba31d297787726444"
server_key_iv: "3df7fd318e24972827c2334815482fb8"
server_key_version: "encryptionkey@0"
I20260810 07:19:51.508546 2877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:51.527773 2877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:51.531268 2877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:51.531486 2877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:51.532269 2877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:51.533270 2877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:51.533320 2877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.533388 2877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:51.533425 2877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:51.544442 2877 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:36055
I20260810 07:19:51.544464 3072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:36055 every 8 connection(s)
I20260810 07:19:51.545503 2877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:51.550160 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 2877
I20260810 07:19:51.550307 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:51.550632 1527 external_mini_cluster.cc:1428] Setting key cc2d0ff0f2f31cc18937035dad584e6e
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[2300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:51.565344 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.547543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:60023 (local address 127.1.125.254:39825)
0810 07:19:51.547720 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.547726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.548672 (+ 946us) server_negotiation.cc:407] Connection header received
0810 07:19:51.550234 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.550239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.550306 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.550409 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.551739 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.552428 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.553681 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.553921 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.554035 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.561711 (+ 7676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.561715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.561717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.561744 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.563128 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.563692 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.563701 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.563702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.563782 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.564165 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.564169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.564170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.564376 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.564519 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.565052 (+ 533us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.565177 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 07:19:51.566345 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.547972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39825 (local address 127.1.125.195:60023)
0810 07:19:51.548444 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:51.548548 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:51.549959 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:51.550609 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:51.550615 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:51.551066 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:51.551582 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.551598 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.552623 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.552627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.553548 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.553558 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.554079 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.554083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.554273 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.559858 (+ 5585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:51.559877 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:51.561513 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:51.563263 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:51.563270 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:51.563285 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:51.563573 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:51.563880 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:51.563883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:51.563885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:51.564053 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:51.564525 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:51.564534 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:51.564802 (+ 268us) client_negotiation.cc:769] Sending connection context
0810 07:19:51.565011 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 07:19:51.565367 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":115,"threads_started":1}
I20260810 07:19:51.567468 3073 heartbeater.cc:347] Connected to a master server at 127.1.125.254:39825
I20260810 07:19:51.567742 3073 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:51.568259 3073 heartbeater.cc:511] Master 127.1.125.254:39825 requested a full tablet report, sending...
I20260810 07:19:51.569350 2371 ts_manager.cc:194] Registered new tserver with Master: 7b7117c50b274d27b86a94dc730b2208 (127.1.125.195:36055)
I20260810 07:19:51.569926 2371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:60023
I20260810 07:19:51.576402 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:19:51.587551 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.579877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54898 (local address 127.1.125.254:39825)
0810 07:19:51.580033 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.580037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.580205 (+ 168us) server_negotiation.cc:407] Connection header received
0810 07:19:51.580410 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.580414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.580463 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.580592 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.581353 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.581883 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.582811 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.583058 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.583169 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.584606 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.584610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.584613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.584635 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.585809 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.586248 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.586253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.586255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.586342 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.586632 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.586635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.586637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.586860 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.587007 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.587264 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.587384 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:19:51.592242 2371 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:51.595569 2371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:51.611230 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.605810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:35055 (local address 127.0.0.1:36070)
0810 07:19:51.606693 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:51.606710 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:51.606792 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:51.607305 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:51.607308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:51.607326 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:51.607425 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.607431 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.608461 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.608464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.609684 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.609695 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.610627 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.610631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.610757 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.610798 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 07:19:51.610918 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 07:19:51.611037 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":84,"thread_start_us":100,"threads_started":1}
I20260810 07:19:51.611244 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.606255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36070 (local address 127.1.125.194:35055)
0810 07:19:51.606643 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.606661 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.606885 (+ 224us) server_negotiation.cc:407] Connection header received
0810 07:19:51.606968 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.606973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.607164 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.607326 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:51.607527 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.608282 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.609820 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.610496 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.610627 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.610773 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.610969 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.611071 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":142,"threads_started":1}
I20260810 07:19:51.611869 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.604926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:44405 (local address 127.0.0.1:57020)
0810 07:19:51.606028 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:51.606043 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:51.606150 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:51.606928 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:51.606931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:51.606947 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:51.607050 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.607056 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.608498 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.608502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.610189 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.610201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.611414 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.611418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.611526 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.611554 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 07:19:51.611661 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 07:19:51.611733 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":260,"thread_start_us":110,"threads_started":1}
I20260810 07:19:51.611929 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.605117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57020 (local address 127.1.125.193:44405)
0810 07:19:51.605875 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.605884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.606262 (+ 378us) server_negotiation.cc:407] Connection header received
0810 07:19:51.606355 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.606360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.606598 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.606752 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:51.607160 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.608336 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.610341 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.611097 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.611196 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.611346 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.611697 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.611787 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":131,"threads_started":1}
I20260810 07:19:51.612164 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.604791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:36055 (local address 127.0.0.1:40144)
0810 07:19:51.605783 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:51.605833 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:51.605957 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:51.606777 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:51.606783 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:51.606808 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:51.606948 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.606965 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.608475 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.608505 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.610191 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.610201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.611560 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.611564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.611667 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.611691 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:19:51.611767 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:19:51.611848 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"mutex_wait_us":210,"thread_start_us":88,"threads_started":1}
I20260810 07:19:51.612208 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.604922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40144 (local address 127.1.125.195:36055)
0810 07:19:51.606302 (+ 1380us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.606311 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.606331 (+ 20us) server_negotiation.cc:407] Connection header received
0810 07:19:51.606396 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.606400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.606620 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.606787 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:51.607089 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.608308 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.610328 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.611136 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.611247 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.611431 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.611947 (+ 516us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.612048 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1271,"thread_start_us":355,"threads_started":1}
I20260810 07:19:51.614966 2559 tablet_service.cc:1467] Processing CreateTablet for tablet 86b2f316b690492fb42f0dbc026b6aa3 (DEFAULT_TABLE table=test-table [id=ae60c0ba99f14fcea8100a3fa386c65a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:51.615116 2977 tablet_service.cc:1467] Processing CreateTablet for tablet 86b2f316b690492fb42f0dbc026b6aa3 (DEFAULT_TABLE table=test-table [id=ae60c0ba99f14fcea8100a3fa386c65a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:51.615401 2770 tablet_service.cc:1467] Processing CreateTablet for tablet 86b2f316b690492fb42f0dbc026b6aa3 (DEFAULT_TABLE table=test-table [id=ae60c0ba99f14fcea8100a3fa386c65a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:51.616125 2559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86b2f316b690492fb42f0dbc026b6aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:51.616215 2977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86b2f316b690492fb42f0dbc026b6aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:51.616251 2770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86b2f316b690492fb42f0dbc026b6aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:51.624097 3098 tablet_bootstrap.cc:492] T 86b2f316b690492fb42f0dbc026b6aa3 P 0fa33829b5fa4fc59863fe587d8aeff6: Bootstrap starting.
I20260810 07:19:51.624097 3100 tablet_bootstrap.cc:492] T 86b2f316b690492fb42f0dbc026b6aa3 P 31e006e3262948e399f3df0d9b21154c: Bootstrap starting.
I20260810 07:19:51.624810 3102 tablet_bootstrap.cc:492] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208: Bootstrap starting.
I20260810 07:19:51.626703 3102 tablet_bootstrap.cc:654] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:51.626745 3098 tablet_bootstrap.cc:654] T 86b2f316b690492fb42f0dbc026b6aa3 P 0fa33829b5fa4fc59863fe587d8aeff6: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:51.627373 3100 tablet_bootstrap.cc:654] T 86b2f316b690492fb42f0dbc026b6aa3 P 31e006e3262948e399f3df0d9b21154c: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:51.627808 3098 log.cc:824] T 86b2f316b690492fb42f0dbc026b6aa3 P 0fa33829b5fa4fc59863fe587d8aeff6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:51.628036 3102 log.cc:824] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:51.628365 3100 log.cc:824] T 86b2f316b690492fb42f0dbc026b6aa3 P 31e006e3262948e399f3df0d9b21154c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:51.630066 3102 tablet_bootstrap.cc:492] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208: No bootstrap required, opened a new log
I20260810 07:19:51.630142 3098 tablet_bootstrap.cc:492] T 86b2f316b690492fb42f0dbc026b6aa3 P 0fa33829b5fa4fc59863fe587d8aeff6: No bootstrap required, opened a new log
I20260810 07:19:51.630223 3102 ts_tablet_manager.cc:1397] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:19:51.630323 3098 ts_tablet_manager.cc:1397] T 86b2f316b690492fb42f0dbc026b6aa3 P 0fa33829b5fa4fc59863fe587d8aeff6: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:19:51.630378 3100 tablet_bootstrap.cc:492] T 86b2f316b690492fb42f0dbc026b6aa3 P 31e006e3262948e399f3df0d9b21154c: No bootstrap required, opened a new log
I20260810 07:19:51.630534 3100 ts_tablet_manager.cc:1397] T 86b2f316b690492fb42f0dbc026b6aa3 P 31e006e3262948e399f3df0d9b21154c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 07:19:51.633234 3102 raft_consensus.cc:348] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7117c50b274d27b86a94dc730b2208" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36055 } } peers { permanent_uuid: "0fa33829b5fa4fc59863fe587d8aeff6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44405 } } peers { permanent_uuid: "31e006e3262948e399f3df0d9b21154c" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35055 } }
I20260810 07:19:51.633543 3102 raft_consensus.cc:374] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:51.633612 3102 raft_consensus.cc:729] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b7117c50b274d27b86a94dc730b2208, State: Initialized, Role: FOLLOWER
I20260810 07:19:51.634171 3102 consensus_queue.cc:260] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7117c50b274d27b86a94dc730b2208" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36055 } } peers { permanent_uuid: "0fa33829b5fa4fc59863fe587d8aeff6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44405 } } peers { permanent_uuid: "31e006e3262948e399f3df0d9b21154c" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35055 } }
I20260810 07:19:51.634434 3109 raft_consensus.cc:482] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:51.634644 3109 raft_consensus.cc:504] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7117c50b274d27b86a94dc730b2208" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36055 } } peers { permanent_uuid: "0fa33829b5fa4fc59863fe587d8aeff6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44405 } } peers { permanent_uuid: "31e006e3262948e399f3df0d9b21154c" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35055 } }
I20260810 07:19:51.634593 3100 raft_consensus.cc:348] T 86b2f316b690492fb42f0dbc026b6aa3 P 31e006e3262948e399f3df0d9b21154c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7117c50b274d27b86a94dc730b2208" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36055 } } peers { permanent_uuid: "0fa33829b5fa4fc59863fe587d8aeff6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44405 } } peers { permanent_uuid: "31e006e3262948e399f3df0d9b21154c" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35055 } }
I20260810 07:19:51.634881 3100 raft_consensus.cc:374] T 86b2f316b690492fb42f0dbc026b6aa3 P 31e006e3262948e399f3df0d9b21154c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:51.634950 3100 raft_consensus.cc:729] T 86b2f316b690492fb42f0dbc026b6aa3 P 31e006e3262948e399f3df0d9b21154c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e006e3262948e399f3df0d9b21154c, State: Initialized, Role: FOLLOWER
I20260810 07:19:51.635267 3102 ts_tablet_manager.cc:1428] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:19:51.635658 3073 heartbeater.cc:503] Master 127.1.125.254:39825 was elected leader, sending a full tablet report...
I20260810 07:19:51.635857 3098 raft_consensus.cc:348] T 86b2f316b690492fb42f0dbc026b6aa3 P 0fa33829b5fa4fc59863fe587d8aeff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7117c50b274d27b86a94dc730b2208" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36055 } } peers { permanent_uuid: "0fa33829b5fa4fc59863fe587d8aeff6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44405 } } peers { permanent_uuid: "31e006e3262948e399f3df0d9b21154c" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35055 } }
I20260810 07:19:51.636121 3098 raft_consensus.cc:374] T 86b2f316b690492fb42f0dbc026b6aa3 P 0fa33829b5fa4fc59863fe587d8aeff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:51.636170 3098 raft_consensus.cc:729] T 86b2f316b690492fb42f0dbc026b6aa3 P 0fa33829b5fa4fc59863fe587d8aeff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa33829b5fa4fc59863fe587d8aeff6, State: Initialized, Role: FOLLOWER
I20260810 07:19:51.636618 3098 consensus_queue.cc:260] T 86b2f316b690492fb42f0dbc026b6aa3 P 0fa33829b5fa4fc59863fe587d8aeff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7117c50b274d27b86a94dc730b2208" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36055 } } peers { permanent_uuid: "0fa33829b5fa4fc59863fe587d8aeff6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44405 } } peers { permanent_uuid: "31e006e3262948e399f3df0d9b21154c" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35055 } }
I20260810 07:19:51.636817 3109 leader_election.cc:288] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa33829b5fa4fc59863fe587d8aeff6 (127.1.125.193:44405), 31e006e3262948e399f3df0d9b21154c (127.1.125.194:35055)
I20260810 07:19:51.637568 3098 ts_tablet_manager.cc:1428] T 86b2f316b690492fb42f0dbc026b6aa3 P 0fa33829b5fa4fc59863fe587d8aeff6: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 07:19:51.637915 2662 heartbeater.cc:503] Master 127.1.125.254:39825 was elected leader, sending a full tablet report...
I20260810 07:19:51.635430 3100 consensus_queue.cc:260] T 86b2f316b690492fb42f0dbc026b6aa3 P 31e006e3262948e399f3df0d9b21154c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7117c50b274d27b86a94dc730b2208" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36055 } } peers { permanent_uuid: "0fa33829b5fa4fc59863fe587d8aeff6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44405 } } peers { permanent_uuid: "31e006e3262948e399f3df0d9b21154c" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35055 } }
I20260810 07:19:51.639611 3100 ts_tablet_manager.cc:1428] T 86b2f316b690492fb42f0dbc026b6aa3 P 31e006e3262948e399f3df0d9b21154c: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260810 07:19:51.639933 2871 heartbeater.cc:503] Master 127.1.125.254:39825 was elected leader, sending a full tablet report...
I20260810 07:19:51.641690 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.636271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:44405 (local address 127.1.125.195:36165)
0810 07:19:51.636692 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:51.636711 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:51.636801 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:51.637656 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:51.637659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:51.637673 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:51.637794 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.637800 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.639105 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.639108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.640287 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.640297 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.641218 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.641220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.641306 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.641353 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 07:19:51.641450 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 07:19:51.641527 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281}
I20260810 07:19:51.641963 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.636967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:36165 (local address 127.1.125.193:44405)
0810 07:19:51.637445 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.637450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.637464 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:19:51.637512 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.637515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.637566 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.637635 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:51.638261 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.638964 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.640455 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.641110 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.641236 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.641294 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.641771 (+ 477us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.641839 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375}
I20260810 07:19:51.642730 2590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b2f316b690492fb42f0dbc026b6aa3" candidate_uuid: "7b7117c50b274d27b86a94dc730b2208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa33829b5fa4fc59863fe587d8aeff6" is_pre_election: true
I20260810 07:19:51.643004 2590 raft_consensus.cc:2454] T 86b2f316b690492fb42f0dbc026b6aa3 P 0fa33829b5fa4fc59863fe587d8aeff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b7117c50b274d27b86a94dc730b2208 in term 0.
I20260810 07:19:51.643513 2909 leader_election.cc:302] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa33829b5fa4fc59863fe587d8aeff6, 7b7117c50b274d27b86a94dc730b2208; no voters:
I20260810 07:19:51.643812 3110 raft_consensus.cc:2781] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:51.643930 3110 raft_consensus.cc:482] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:51.644026 3110 raft_consensus.cc:3037] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:51.645303 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.636429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:38755 (local address 127.1.125.194:35055)
0810 07:19:51.636783 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.636788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.637460 (+ 672us) server_negotiation.cc:407] Connection header received
0810 07:19:51.637984 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.637987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.638048 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.638105 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:51.641703 (+ 3598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.642396 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.644023 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.644658 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.644754 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.644799 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.645149 (+ 350us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.645214 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 07:19:51.645398 3113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.636271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:35055 (local address 127.1.125.195:38755)
0810 07:19:51.637372 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:51.637387 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:51.637523 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:51.641401 (+ 3878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:51.641404 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:51.641424 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:51.641555 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.641563 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.642565 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.642568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.643869 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:51.643880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.644915 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.644919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:51.645019 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.645041 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:19:51.645124 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 07:19:51.645193 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1027,"mutex_wait_us":829,"thread_start_us":123,"threads_started":1}
I20260810 07:19:51.645498 3110 raft_consensus.cc:504] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7117c50b274d27b86a94dc730b2208" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36055 } } peers { permanent_uuid: "0fa33829b5fa4fc59863fe587d8aeff6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44405 } } peers { permanent_uuid: "31e006e3262948e399f3df0d9b21154c" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35055 } }
I20260810 07:19:51.645874 3110 leader_election.cc:288] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa33829b5fa4fc59863fe587d8aeff6 (127.1.125.193:44405), 31e006e3262948e399f3df0d9b21154c (127.1.125.194:35055)
I20260810 07:19:51.646245 2590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b2f316b690492fb42f0dbc026b6aa3" candidate_uuid: "7b7117c50b274d27b86a94dc730b2208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa33829b5fa4fc59863fe587d8aeff6"
I20260810 07:19:51.646380 2590 raft_consensus.cc:3037] T 86b2f316b690492fb42f0dbc026b6aa3 P 0fa33829b5fa4fc59863fe587d8aeff6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:51.646385 2799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b2f316b690492fb42f0dbc026b6aa3" candidate_uuid: "7b7117c50b274d27b86a94dc730b2208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e006e3262948e399f3df0d9b21154c" is_pre_election: true
I20260810 07:19:51.646454 2796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b2f316b690492fb42f0dbc026b6aa3" candidate_uuid: "7b7117c50b274d27b86a94dc730b2208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e006e3262948e399f3df0d9b21154c"
I20260810 07:19:51.646600 2796 raft_consensus.cc:3037] T 86b2f316b690492fb42f0dbc026b6aa3 P 31e006e3262948e399f3df0d9b21154c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:51.647619 2590 raft_consensus.cc:2454] T 86b2f316b690492fb42f0dbc026b6aa3 P 0fa33829b5fa4fc59863fe587d8aeff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b7117c50b274d27b86a94dc730b2208 in term 1.
I20260810 07:19:51.647934 2909 leader_election.cc:302] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa33829b5fa4fc59863fe587d8aeff6, 7b7117c50b274d27b86a94dc730b2208; no voters:
I20260810 07:19:51.647923 2796 raft_consensus.cc:2454] T 86b2f316b690492fb42f0dbc026b6aa3 P 31e006e3262948e399f3df0d9b21154c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b7117c50b274d27b86a94dc730b2208 in term 1.
I20260810 07:19:51.648092 3110 raft_consensus.cc:2781] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:51.648208 3110 raft_consensus.cc:686] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [term 1 LEADER]: Becoming Leader. State: Replica: 7b7117c50b274d27b86a94dc730b2208, State: Running, Role: LEADER
I20260810 07:19:51.648576 3110 consensus_queue.cc:237] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7117c50b274d27b86a94dc730b2208" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36055 } } peers { permanent_uuid: "0fa33829b5fa4fc59863fe587d8aeff6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44405 } } peers { permanent_uuid: "31e006e3262948e399f3df0d9b21154c" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35055 } }
I20260810 07:19:51.651693 2370 catalog_manager.cc:5495] T 86b2f316b690492fb42f0dbc026b6aa3 P 7b7117c50b274d27b86a94dc730b2208 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b7117c50b274d27b86a94dc730b2208 (127.1.125.195). New cstate: current_term: 1 leader_uuid: "7b7117c50b274d27b86a94dc730b2208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7117c50b274d27b86a94dc730b2208" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa33829b5fa4fc59863fe587d8aeff6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31e006e3262948e399f3df0d9b21154c" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35055 } health_report { overall_health: UNKNOWN } } }
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[2300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.193@KRBTEST.COM
I20260810 07:19:51.664629 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.655637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57022 (local address 127.1.125.193:44405)
0810 07:19:51.655760 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.655764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.656116 (+ 352us) server_negotiation.cc:407] Connection header received
0810 07:19:51.656191 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.656194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.656234 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.656298 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.657066 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.657548 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.658429 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.658634 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.658710 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.661373 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.661383 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.661396 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.661422 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.662918 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.663342 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.663349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.663350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.663431 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.663677 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.663679 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.663681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.663916 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.664068 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.664345 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.664446 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[2300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[2300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.125.193@KRBTEST.COM
I20260810 07:19:51.694763 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.686477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57024 (local address 127.1.125.193:44405)
0810 07:19:51.686659 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.686663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.686798 (+ 135us) server_negotiation.cc:407] Connection header received
0810 07:19:51.686915 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.686919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.686973 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.687061 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.687685 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.688185 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.689073 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.689297 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.689376 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.691821 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.691824 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.691826 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.691849 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.693172 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.693577 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.693581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.693584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.693686 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.693934 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.693936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.693937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.694105 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.694219 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.694445 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.694543 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[2300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[2300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.193@KRBTEST.COM
I20260810 07:19:51.722303 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:51.713936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57038 (local address 127.1.125.193:44405)
0810 07:19:51.714126 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:51.714129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:51.714208 (+ 79us) server_negotiation.cc:407] Connection header received
0810 07:19:51.714320 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:51.714323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:51.714365 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:51.714436 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:51.715069 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.715555 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.716336 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:51.716606 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:51.716685 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:51.719103 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:51.719106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:51.719108 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:51.719132 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:51.720647 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.721030 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.721032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.721034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.721110 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:51.721342 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:51.721344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:51.721345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:51.721571 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:51.721712 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:51.721919 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 07:19:51.722069 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260810 07:19:51.723753 2529 tablet_service.cc:2908] Rejecting scan request for tablet 86b2f316b690492fb42f0dbc026b6aa3: Uninitialized: clean time has not yet been initialized
I20260810 07:19:51.726130 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 2470
I20260810 07:19:51.743525 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 2670
I20260810 07:19:51.762635 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 2877
I20260810 07:19:51.771351 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 2320
2026-08-10T07:19:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2628 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[3131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[3131](info): set up 2 sockets
Aug 10 07:19:51 dist-test-slave-44d2 krb5kdc[3131](info): commencing operation
krb5kdc: starting...
W20260810 07:19:53.087224 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:19:53 dist-test-slave-44d2 krb5kdc[3131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:19:53.219555 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:39029
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:38539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:39029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:53.337998 3152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:53.338274 3152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:53.338362 3152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:53.342442 3152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:53.342492 3152 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 07:19:53.342509 3152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:53.342527 3152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:53.342543 3152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:53.342603 3152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:53.347772 3152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:38539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:39029
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:39029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:53.348838 3152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:53.349872 3152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:53.358703 3166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:53.358889 3152 server_base.cc:1034] running on GCE node
W20260810 07:19:53.358681 3164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:53.358676 3159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:53.359577 3152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:53.360764 3152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:53.361960 3152 hybrid_clock.cc:648] HybridClock initialized: now 1786346393361938 us; error 32 us; skew 500 ppm
Aug 10 07:19:53 dist-test-slave-44d2 krb5kdc[3131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:53.364794 3152 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:19:53.365738 3152 webserver.cc:460] Webserver started at http://127.1.125.254:44367/ using document root <none> and password file <none>
I20260810 07:19:53.366309 3152 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:53.366406 3152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:53.366648 3152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:53.368764 3152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "d21e92387112427b854a4d30fbfcd260"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-44d2"
server_key: "a107370a713350d1abb50fb277f3d1c9"
server_key_iv: "74bdad88b26b49c4470a460614f00d84"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.369339 3152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "d21e92387112427b854a4d30fbfcd260"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-44d2"
server_key: "a107370a713350d1abb50fb277f3d1c9"
server_key_iv: "74bdad88b26b49c4470a460614f00d84"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.373050 3152 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 07:19:53.375480 3178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:53.376848 3152 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:53.377008 3152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "d21e92387112427b854a4d30fbfcd260"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-44d2"
server_key: "a107370a713350d1abb50fb277f3d1c9"
server_key_iv: "74bdad88b26b49c4470a460614f00d84"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.377121 3152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:53.399248 3152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:53.402732 3152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:53.402956 3152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:53.411224 3152 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:39029
I20260810 07:19:53.411280 3277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:39029 every 8 connection(s)
I20260810 07:19:53.412221 3152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:53.415130 3278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:53.416734 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 3152
I20260810 07:19:53.416864 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:19:53.417135 1527 external_mini_cluster.cc:1428] Setting key 8b2d1d205b197afb819f25985dd9fbe3
I20260810 07:19:53.421972 3278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260: Bootstrap starting.
I20260810 07:19:53.424721 3278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:53.425468 3278 log.cc:824] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260: Log is configured to *not* fsync() on all Append() calls
Aug 10 07:19:53 dist-test-slave-44d2 krb5kdc[3131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:53.427815 3278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260: No bootstrap required, opened a new log
I20260810 07:19:53.432070 3278 raft_consensus.cc:348] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d21e92387112427b854a4d30fbfcd260" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39029 } }
I20260810 07:19:53.432296 3278 raft_consensus.cc:374] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:53.432334 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:53.418584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45296 (local address 127.1.125.254:39029)
0810 07:19:53.419011 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:53.419031 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:53.419077 (+ 46us) server_negotiation.cc:407] Connection header received
0810 07:19:53.419967 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:53.419993 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:53.420424 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:53.420841 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:53.421522 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.422980 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.423809 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.424511 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.424713 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:53.427616 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:53.427621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:53.427634 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:53.427661 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:53.429446 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.429969 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.429974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.429979 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.430088 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.430402 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.430404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.430405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.430791 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:53.430938 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:53.431260 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 07:19:53.431578 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":109,"threads_started":1}
I20260810 07:19:53.432363 3278 raft_consensus.cc:729] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d21e92387112427b854a4d30fbfcd260, State: Initialized, Role: FOLLOWER
I20260810 07:19:53.432852 3278 consensus_queue.cc:260] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d21e92387112427b854a4d30fbfcd260" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39029 } }
I20260810 07:19:53.433035 3278 raft_consensus.cc:388] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:53.433121 3278 raft_consensus.cc:482] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:53.433251 3278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:53.434386 3278 raft_consensus.cc:504] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d21e92387112427b854a4d30fbfcd260" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39029 } }
I20260810 07:19:53.434747 3278 leader_election.cc:302] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d21e92387112427b854a4d30fbfcd260; no voters:
I20260810 07:19:53.435129 3278 leader_election.cc:288] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:53.435189 3286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:53.435385 3286 raft_consensus.cc:686] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [term 1 LEADER]: Becoming Leader. State: Replica: d21e92387112427b854a4d30fbfcd260, State: Running, Role: LEADER
I20260810 07:19:53.435695 3286 consensus_queue.cc:237] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d21e92387112427b854a4d30fbfcd260" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39029 } }
I20260810 07:19:53.436431 3278 sys_catalog.cc:564] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:53.437546 3287 sys_catalog.cc:455] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d21e92387112427b854a4d30fbfcd260. Latest consensus state: current_term: 1 leader_uuid: "d21e92387112427b854a4d30fbfcd260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d21e92387112427b854a4d30fbfcd260" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39029 } } }
I20260810 07:19:53.437690 3287 sys_catalog.cc:458] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:53.438187 3291 sys_catalog.cc:455] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d21e92387112427b854a4d30fbfcd260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d21e92387112427b854a4d30fbfcd260" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39029 } } }
I20260810 07:19:53.438287 3291 sys_catalog.cc:458] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260 [sys.catalog]: This master's current role is: LEADER
W20260810 07:19:53.441526 3311 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:53.441618 3311 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:53.441690 3301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:53.442444 3301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:53.448585 3301 catalog_manager.cc:1347] Generated new cluster ID: ba8a44f5e2744aa0b104cbd9e64f9c0d
I20260810 07:19:53.448648 3301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:53.472944 3301 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:53.474114 3301 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:53.483876 3301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260: Generated new TSK 0
I20260810 07:19:53.484719 3301 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:19:53.523033 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39029
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:38539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:53.664772 3320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:53.665066 3320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:53.665160 3320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:53.669186 3320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:53.669262 3320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:53.669328 3320 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:19:53.669409 3320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:19:53.674803 3320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:38539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.125.254:39029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:53.675926 3320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:53.677053 3320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:53.687258 3327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:53.687206 3326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:53.687305 3329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:53.687885 3320 server_base.cc:1034] running on GCE node
I20260810 07:19:53.688304 3320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:53.689081 3320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:53.690272 3320 hybrid_clock.cc:648] HybridClock initialized: now 1786346393690242 us; error 50 us; skew 500 ppm
Aug 10 07:19:53 dist-test-slave-44d2 krb5kdc[3131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:53.693567 3320 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:19:53.694836 3320 webserver.cc:460] Webserver started at http://127.1.125.193:37761/ using document root <none> and password file <none>
I20260810 07:19:53.695623 3320 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:53.695722 3320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:53.695993 3320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:53.699182 3320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "94254137e0f447b1bb0fcb4006a2ba93"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-44d2"
server_key: "44719adeeffa6468c54d4f2ed2d66680"
server_key_iv: "e1635dfc014bc6873ba8079d4ce6872d"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.699955 3320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "94254137e0f447b1bb0fcb4006a2ba93"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-44d2"
server_key: "44719adeeffa6468c54d4f2ed2d66680"
server_key_iv: "e1635dfc014bc6873ba8079d4ce6872d"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.705782 3320 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 07:19:53.709338 3341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:53.711148 3320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.001s
I20260810 07:19:53.711298 3320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "94254137e0f447b1bb0fcb4006a2ba93"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-44d2"
server_key: "44719adeeffa6468c54d4f2ed2d66680"
server_key_iv: "e1635dfc014bc6873ba8079d4ce6872d"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.711458 3320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:53.745811 3320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:53.750465 3320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:53.750728 3320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:53.753160 3320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:53.753257 3320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:53.753357 3320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:53.753403 3320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:19:53 dist-test-slave-44d2 krb5kdc[3131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:53.765693 3320 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:45291
I20260810 07:19:53.766482 3515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:45291 every 8 connection(s)
I20260810 07:19:53.767696 3320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:53.771026 3354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:53.754113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39029 (local address 127.1.125.193:57749)
0810 07:19:53.754592 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:53.754646 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:53.755697 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:53.756275 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:53.756282 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:53.756750 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:53.757409 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:53.757435 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.758689 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.758692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:53.759583 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:53.759593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.760290 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.760295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:53.760556 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:53.761514 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:53.761543 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:53.764701 (+ 3158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:53.767016 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:53.767025 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:53.767044 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:53.767470 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:53.767884 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:53.767890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:53.767895 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:53.768094 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:53.768823 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:53.768833 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:53.769180 (+ 347us) client_negotiation.cc:769] Sending connection context
0810 07:19:53.769441 (+ 261us) client_negotiation.cc:240] Negotiation successful
0810 07:19:53.769841 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":136,"threads_started":1}
I20260810 07:19:53.771535 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 3320
I20260810 07:19:53.771663 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:53.771934 1527 external_mini_cluster.cc:1428] Setting key 6e5bb0f4c5d04e42ef676504f8fc4caa
I20260810 07:19:53.772030 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:53.753578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:57749 (local address 127.1.125.254:39029)
0810 07:19:53.753710 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:53.753715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:53.754753 (+ 1038us) server_negotiation.cc:407] Connection header received
0810 07:19:53.755917 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:53.755920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:53.755982 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:53.756090 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:53.757616 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.758555 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.759724 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.760125 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.760239 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:53.764947 (+ 4708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:53.764954 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:53.764960 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:53.765008 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:53.766892 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.767629 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.767637 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.767641 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.767754 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.768230 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.768237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.768240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.768630 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:53.768753 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:53.771635 (+ 2882us) server_negotiation.cc:299] Negotiation successful
0810 07:19:53.771800 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:19:53.773699 3516 heartbeater.cc:347] Connected to a master server at 127.1.125.254:39029
I20260810 07:19:53.773901 3516 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:53.774557 3516 heartbeater.cc:511] Master 127.1.125.254:39029 requested a full tablet report, sending...
I20260810 07:19:53.776384 3204 ts_manager.cc:194] Registered new tserver with Master: 94254137e0f447b1bb0fcb4006a2ba93 (127.1.125.193:45291)
I20260810 07:19:53.778080 3204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:57749
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
I20260810 07:19:53.788656 3526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:53.776570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39029 (local address 127.1.125.193:55091)
0810 07:19:53.779237 (+ 2667us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:53.779255 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:53.779418 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:53.779863 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:53.779869 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:53.780133 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:53.780301 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:53.780361 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.781618 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.781624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:53.782667 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:53.782683 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.783433 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.783437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:53.783559 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:53.784219 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:53.784241 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:53.784768 (+ 527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:53.786831 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:53.786834 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:53.786836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:53.787040 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:53.787365 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:53.787368 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:53.787369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:53.787476 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:53.787927 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:53.787930 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:53.788049 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 07:19:53.788162 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 07:19:53.788349 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2558,"thread_start_us":108,"threads_started":1}
I20260810 07:19:53.788990 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:53.776734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:55091 (local address 127.1.125.254:39029)
0810 07:19:53.776890 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:53.776896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:53.779327 (+ 2431us) server_negotiation.cc:407] Connection header received
0810 07:19:53.779545 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:53.779550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:53.779607 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:53.779672 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:53.780549 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.781396 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.782816 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.783266 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.783413 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:53.784952 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:53.784959 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:53.784965 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:53.785002 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:53.786701 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.787158 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.787162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.787163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.787237 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.787586 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.787590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.787591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.787776 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:53.787884 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:53.788602 (+ 718us) server_negotiation.cc:299] Negotiation successful
0810 07:19:53.788729 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:19:53.806161 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39029
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:38539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:53.925462 3529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:53.925729 3529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:53.925817 3529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:53.930148 3529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:53.930217 3529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:53.930296 3529 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:19:53.930384 3529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:19:53.935729 3529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:38539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.125.254:39029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:53.936962 3529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:53.938256 3529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:53.948446 3529 server_base.cc:1034] running on GCE node
W20260810 07:19:53.948385 3538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:53.948411 3537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:53.948405 3541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:53.949074 3529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:53.949692 3529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:53.950863 3529 hybrid_clock.cc:648] HybridClock initialized: now 1786346393950842 us; error 33 us; skew 500 ppm
Aug 10 07:19:53 dist-test-slave-44d2 krb5kdc[3131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:53.953792 3529 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:19:53.954754 3529 webserver.cc:460] Webserver started at http://127.1.125.194:44497/ using document root <none> and password file <none>
I20260810 07:19:53.955291 3529 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:53.955350 3529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:53.955534 3529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:53.957684 3529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "31a49ba49aa84fe0a6770a4f2bc255c4"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-44d2"
server_key: "491a395c7e3d14bb99e9f957831bafcf"
server_key_iv: "06d94d7c118f690a5f873b047437d1cb"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.958200 3529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "31a49ba49aa84fe0a6770a4f2bc255c4"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-44d2"
server_key: "491a395c7e3d14bb99e9f957831bafcf"
server_key_iv: "06d94d7c118f690a5f873b047437d1cb"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.961892 3529 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:19:53.964325 3551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:53.965615 3529 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:53.965745 3529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "31a49ba49aa84fe0a6770a4f2bc255c4"
format_stamp: "Formatted at 2026-08-10 07:19:53 on dist-test-slave-44d2"
server_key: "491a395c7e3d14bb99e9f957831bafcf"
server_key_iv: "06d94d7c118f690a5f873b047437d1cb"
server_key_version: "encryptionkey@0"
I20260810 07:19:53.965850 3529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:53.979743 3529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:53.983134 3529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:53.983340 3529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:53.985474 3529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:53.985553 3529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:53.985606 3529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:53.985638 3529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:19:53 dist-test-slave-44d2 krb5kdc[3131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:53.996608 3529 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:34951
I20260810 07:19:53.996647 3729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:34951 every 8 connection(s)
I20260810 07:19:53.997797 3529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:53.999806 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:53.986219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:46911 (local address 127.1.125.254:39029)
0810 07:19:53.986362 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:53.986367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:53.987672 (+ 1305us) server_negotiation.cc:407] Connection header received
0810 07:19:53.988879 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:53.988882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:53.988945 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:53.989081 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:53.990288 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.990825 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.991989 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.992277 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.992394 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:53.995874 (+ 3480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:53.995877 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:53.995879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:53.995905 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:53.997618 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.998237 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.998242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.998245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.998347 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:53.998780 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:53.998786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:53.998789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:53.999001 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:53.999082 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:53.999584 (+ 502us) server_negotiation.cc:299] Negotiation successful
0810 07:19:53.999687 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:19:54.000622 3563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:53.987037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39029 (local address 127.1.125.194:46911)
0810 07:19:53.987504 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:53.987556 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:53.988668 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:53.989261 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:53.989268 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:53.989603 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:53.990142 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:53.990156 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.990957 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.990960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:53.991859 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:53.991869 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:53.992447 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:53.992451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:53.992692 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:53.993324 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:53.993341 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:53.995697 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:53.997750 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:53.997755 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:53.997766 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:53.998101 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:53.998453 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:53.998456 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:53.998458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:53.998653 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:53.999115 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:53.999119 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:53.999338 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 07:19:53.999498 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 07:19:53.999740 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":137,"threads_started":1}
I20260810 07:19:54.001772 3732 heartbeater.cc:347] Connected to a master server at 127.1.125.254:39029
I20260810 07:19:54.001946 3732 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:54.002434 3732 heartbeater.cc:511] Master 127.1.125.254:39029 requested a full tablet report, sending...
I20260810 07:19:54.003533 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 3529
I20260810 07:19:54.003651 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:54.003640 3203 ts_manager.cc:194] Registered new tserver with Master: 31a49ba49aa84fe0a6770a4f2bc255c4 (127.1.125.194:34951)
I20260810 07:19:54.003942 1527 external_mini_cluster.cc:1428] Setting key 6330137654173e91b3c3d37da93185e5
I20260810 07:19:54.004384 3203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:46911
I20260810 07:19:54.012110 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.004597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:49649 (local address 127.1.125.254:39029)
0810 07:19:54.004736 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.004742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.004996 (+ 254us) server_negotiation.cc:407] Connection header received
0810 07:19:54.005170 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.005173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.005220 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.005305 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.005801 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.006271 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.007378 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.007584 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.007659 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.009017 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.009023 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.009027 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.009057 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.010386 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.010830 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.010834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.010836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.010916 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.011239 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.011242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.011244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.011452 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.011570 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.011833 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.011953 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 07:19:54.012120 3741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.004632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39029 (local address 127.1.125.194:49649)
0810 07:19:54.004896 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.004908 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.005024 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.005341 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.005344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.005571 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.005686 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.005694 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.006402 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.006406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.007256 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.007272 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.007702 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.007708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.007843 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.008317 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.008337 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.008844 (+ 507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.010512 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.010517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.010521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.010727 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.011043 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.011046 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.011048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.011135 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.011562 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.011566 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.011663 (+ 97us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.011778 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.011917 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:19:54.035118 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39029
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:38539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:54.159111 3744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:54.159359 3744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:54.159403 3744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:54.163571 3744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:54.163636 3744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:54.163674 3744 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:19:54.163707 3744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:19:54.169035 3744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:38539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.125.254:39029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:54.170203 3744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:54.171310 3744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:54.181676 3763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:54.181686 3761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:54.181782 3752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:54.182102 3744 server_base.cc:1034] running on GCE node
I20260810 07:19:54.182723 3744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:54.183372 3744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:54.184571 3744 hybrid_clock.cc:648] HybridClock initialized: now 1786346394184553 us; error 27 us; skew 500 ppm
Aug 10 07:19:54 dist-test-slave-44d2 krb5kdc[3131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:54.187602 3744 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:19:54.188508 3744 webserver.cc:460] Webserver started at http://127.1.125.195:37903/ using document root <none> and password file <none>
I20260810 07:19:54.189162 3744 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:54.189229 3744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:54.189476 3744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:54.191570 3744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "817d027e6b1a425786bba9bdb35e8e53"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-44d2"
server_key: "3300747a5755a6c4740e35d0966bf495"
server_key_iv: "31565ceec5d2b87192d445def217fd9c"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.192305 3744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "817d027e6b1a425786bba9bdb35e8e53"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-44d2"
server_key: "3300747a5755a6c4740e35d0966bf495"
server_key_iv: "31565ceec5d2b87192d445def217fd9c"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.199838 3744 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.001s
I20260810 07:19:54.202812 3773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.204195 3744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:19:54.204301 3744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "817d027e6b1a425786bba9bdb35e8e53"
format_stamp: "Formatted at 2026-08-10 07:19:54 on dist-test-slave-44d2"
server_key: "3300747a5755a6c4740e35d0966bf495"
server_key_iv: "31565ceec5d2b87192d445def217fd9c"
server_key_version: "encryptionkey@0"
I20260810 07:19:54.204397 3744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:54.220238 3744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:54.223654 3744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:54.223817 3744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:54.225422 3744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:54.225529 3744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:54.225606 3744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:54.225662 3744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:19:54 dist-test-slave-44d2 krb5kdc[3131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:54.238281 3744 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:33975
I20260810 07:19:54.238292 3959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:33975 every 8 connection(s)
I20260810 07:19:54.239457 3744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:54.240762 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.226345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:38797 (local address 127.1.125.254:39029)
0810 07:19:54.226543 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.226550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.227502 (+ 952us) server_negotiation.cc:407] Connection header received
0810 07:19:54.228888 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.228892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.229052 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.229123 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.231057 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.231680 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.233115 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.233400 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.233511 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.236443 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.236447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.236449 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.236512 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.238455 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.238965 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.238969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.238971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.239078 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.239422 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.239429 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.239434 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.239682 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.239818 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.240511 (+ 693us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.240640 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 07:19:54.241654 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.226777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39029 (local address 127.1.125.195:38797)
0810 07:19:54.227310 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.227370 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.228603 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.229609 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.229620 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.230113 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.230874 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.230898 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.231829 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.231832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.232980 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.232990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.233557 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.233562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.233742 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.234356 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.234378 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.236221 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.238570 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.238574 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.238585 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.238854 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.239186 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.239188 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.239189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.239320 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.239807 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.239830 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.240205 (+ 375us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.240421 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.240750 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":125,"threads_started":1}
I20260810 07:19:54.242794 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 3744
I20260810 07:19:54.242933 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:54.243206 1527 external_mini_cluster.cc:1428] Setting key 192a5e507d7f8cee5e241ffabc41debf
I20260810 07:19:54.244551 3961 heartbeater.cc:347] Connected to a master server at 127.1.125.254:39029
I20260810 07:19:54.244700 3961 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:54.245292 3961 heartbeater.cc:511] Master 127.1.125.254:39029 requested a full tablet report, sending...
I20260810 07:19:54.246433 3203 ts_manager.cc:194] Registered new tserver with Master: 817d027e6b1a425786bba9bdb35e8e53 (127.1.125.195:33975)
I20260810 07:19:54.247087 3203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:38797
I20260810 07:19:54.252935 3970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.245529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39029 (local address 127.1.125.195:41285)
0810 07:19:54.245821 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.245833 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.245964 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.246254 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.246257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.246484 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.246622 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.246630 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.247427 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.247431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.248103 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.248112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.248712 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.248718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.248802 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.249335 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.249347 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.249702 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.251487 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.251490 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.251492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.251701 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.251997 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.252000 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.252001 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.252079 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.252416 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.252418 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.252544 (+ 126us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.252668 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.252781 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":112,"threads_started":1}
I20260810 07:19:54.252997 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.245515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:41285 (local address 127.1.125.254:39029)
0810 07:19:54.245634 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.245639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.245927 (+ 288us) server_negotiation.cc:407] Connection header received
0810 07:19:54.246083 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.246086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.246135 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.246225 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.246732 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.247280 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.248227 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.248573 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.248698 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.249860 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.249863 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.249865 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.249956 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.251370 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.251792 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.251795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.251796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.251911 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.252165 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.252167 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.252168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.252331 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.252417 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.252742 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.252851 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:19:54.257105 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:19:54 dist-test-slave-44d2 krb5kdc[3131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:19:54 dist-test-slave-44d2 krb5kdc[3131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:54.285070 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.276876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45306 (local address 127.1.125.254:39029)
0810 07:19:54.277024 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.277028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.277042 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:19:54.277089 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.277092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.277134 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.277210 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.277913 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.278454 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.279295 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.279510 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.279608 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.282111 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.282132 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.282138 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.282169 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.283438 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.283827 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.283830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.283831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.283904 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.284212 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.284214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.284215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.284375 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.284474 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.284754 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.284851 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 07:19:54.287940 3203 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:54.291075 3203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:54.306120 3992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.300727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43838 (local address 127.1.125.193:45291)
0810 07:19:54.301364 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.301370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.301609 (+ 239us) server_negotiation.cc:407] Connection header received
0810 07:19:54.301702 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.301709 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.301884 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.302016 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:54.302449 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.303554 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.304841 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.305409 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.305539 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.305716 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.305875 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.305981 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":92,"threads_started":1}
I20260810 07:19:54.306258 3986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.299967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:34951 (local address 127.0.0.1:56336)
0810 07:19:54.300996 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.301011 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.301094 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.301787 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.301790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.301822 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:54.302022 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.302031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.303242 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.303247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.304678 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.304689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.305800 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.305804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.305914 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.305941 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.305988 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.306060 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":57,"spinlock_wait_cycles":2432,"thread_start_us":101,"threads_started":1}
I20260810 07:19:54.306120 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.300281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:45291 (local address 127.0.0.1:43838)
0810 07:19:54.301486 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.301499 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.301603 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.302189 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.302193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.302215 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:54.302316 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.302322 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.303674 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.303677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.304714 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.304724 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.305540 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.305544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.305661 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.305699 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.305841 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.305968 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1148,"thread_start_us":98,"threads_started":1}
I20260810 07:19:54.306752 3991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.300542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56336 (local address 127.1.125.194:34951)
0810 07:19:54.301212 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.301227 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.301257 (+ 30us) server_negotiation.cc:407] Connection header received
0810 07:19:54.301347 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.301355 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.301617 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.301960 (+ 343us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:54.302141 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.303051 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.304804 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.305651 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.306201 (+ 550us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.306471 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.306523 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.306610 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":124,"threads_started":1}
I20260810 07:19:54.307659 3990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.299769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:33975 (local address 127.0.0.1:51802)
0810 07:19:54.300697 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.300753 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.300885 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.301767 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.301772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.301793 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:54.302022 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.302031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.303586 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.303592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.305809 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.305818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.307259 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.307263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.307370 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.307394 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.307442 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.307507 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":1072,"thread_start_us":92,"threads_started":1}
I20260810 07:19:54.308383 3987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.299940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51802 (local address 127.1.125.195:33975)
0810 07:19:54.301270 (+ 1330us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.301284 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.301310 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:19:54.301392 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.301397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.301606 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.301775 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:54.302191 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.303435 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.305943 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.307111 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.307955 (+ 844us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.308085 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.308188 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.308270 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1169,"thread_start_us":138,"threads_started":1}
I20260810 07:19:54.309790 3630 tablet_service.cc:1467] Processing CreateTablet for tablet 896d5e1842864f56aa96987c02ac6ecb (DEFAULT_TABLE table=test-table [id=b0a548cf308444ffa91582a920771f05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:54.309788 3409 tablet_service.cc:1467] Processing CreateTablet for tablet 896d5e1842864f56aa96987c02ac6ecb (DEFAULT_TABLE table=test-table [id=b0a548cf308444ffa91582a920771f05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:54.310957 3630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896d5e1842864f56aa96987c02ac6ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:54.310957 3409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896d5e1842864f56aa96987c02ac6ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:54.310784 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 896d5e1842864f56aa96987c02ac6ecb (DEFAULT_TABLE table=test-table [id=b0a548cf308444ffa91582a920771f05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:54.311875 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896d5e1842864f56aa96987c02ac6ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:54.318670 3993 tablet_bootstrap.cc:492] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53: Bootstrap starting.
I20260810 07:19:54.319809 3997 tablet_bootstrap.cc:492] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4: Bootstrap starting.
I20260810 07:19:54.320835 3996 tablet_bootstrap.cc:492] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93: Bootstrap starting.
I20260810 07:19:54.321103 3993 tablet_bootstrap.cc:654] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:54.321933 3997 tablet_bootstrap.cc:654] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:54.322261 3993 log.cc:824] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:54.322954 3997 log.cc:824] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:54.323154 3996 tablet_bootstrap.cc:654] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:54.323968 3996 log.cc:824] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:54.324522 3993 tablet_bootstrap.cc:492] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53: No bootstrap required, opened a new log
I20260810 07:19:54.324740 3993 ts_tablet_manager.cc:1397] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:19:54.325168 3997 tablet_bootstrap.cc:492] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4: No bootstrap required, opened a new log
I20260810 07:19:54.325356 3997 ts_tablet_manager.cc:1397] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260810 07:19:54.325913 3996 tablet_bootstrap.cc:492] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93: No bootstrap required, opened a new log
I20260810 07:19:54.326095 3996 ts_tablet_manager.cc:1397] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:19:54.328133 3993 raft_consensus.cc:348] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } }
I20260810 07:19:54.328382 3993 raft_consensus.cc:374] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:54.328439 3993 raft_consensus.cc:729] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 817d027e6b1a425786bba9bdb35e8e53, State: Initialized, Role: FOLLOWER
I20260810 07:19:54.328958 3993 consensus_queue.cc:260] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } }
I20260810 07:19:54.330024 3993 ts_tablet_manager.cc:1428] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:19:54.330001 3961 heartbeater.cc:503] Master 127.1.125.254:39029 was elected leader, sending a full tablet report...
I20260810 07:19:54.329752 3997 raft_consensus.cc:348] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } }
I20260810 07:19:54.330299 3997 raft_consensus.cc:374] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:54.330360 3997 raft_consensus.cc:729] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a49ba49aa84fe0a6770a4f2bc255c4, State: Initialized, Role: FOLLOWER
I20260810 07:19:54.330788 3997 consensus_queue.cc:260] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } }
I20260810 07:19:54.330909 3996 raft_consensus.cc:348] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } }
I20260810 07:19:54.331162 3996 raft_consensus.cc:374] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:54.331223 3996 raft_consensus.cc:729] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94254137e0f447b1bb0fcb4006a2ba93, State: Initialized, Role: FOLLOWER
I20260810 07:19:54.331614 3996 consensus_queue.cc:260] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } }
I20260810 07:19:54.331773 3997 ts_tablet_manager.cc:1428] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:19:54.332118 3732 heartbeater.cc:503] Master 127.1.125.254:39029 was elected leader, sending a full tablet report...
I20260810 07:19:54.332234 3516 heartbeater.cc:503] Master 127.1.125.254:39029 was elected leader, sending a full tablet report...
I20260810 07:19:54.332558 3996 ts_tablet_manager.cc:1428] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93: Time spent starting tablet: real 0.006s user 0.001s sys 0.007s
W20260810 07:19:54.490725 3964 tablet.cc:2369] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:19:54.499210 3734 tablet.cc:2369] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:19:54.520005 3517 tablet.cc:2369] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:54.633047 4005 raft_consensus.cc:482] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:54.633291 4005 raft_consensus.cc:504] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } }
I20260810 07:19:54.634485 4005 leader_election.cc:288] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 817d027e6b1a425786bba9bdb35e8e53 (127.1.125.195:33975), 94254137e0f447b1bb0fcb4006a2ba93 (127.1.125.193:45291)
I20260810 07:19:54.640000 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.634768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:45291 (local address 127.1.125.194:55695)
0810 07:19:54.635113 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.635129 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.635272 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.635573 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.635576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.635606 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:54.635865 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.635873 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.636952 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.636956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.638469 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.638488 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.639373 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.639376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.639548 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.639578 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.639661 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.639793 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":300,"spinlock_wait_cycles":31872,"thread_start_us":75,"threads_started":1}
I20260810 07:19:54.640009 4008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.634751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:33975 (local address 127.1.125.194:50237)
0810 07:19:54.635113 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.635129 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.635272 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.635573 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.635576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.635601 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:54.635873 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.635879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.636952 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.636956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.638469 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.638481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.639362 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.639365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.639504 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.639578 (+ 74us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.639661 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.639793 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":244,"spinlock_wait_cycles":17536,"thread_start_us":109,"threads_started":1}
I20260810 07:19:54.640021 3987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.634892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:50237 (local address 127.1.125.195:33975)
0810 07:19:54.635080 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.635085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.635221 (+ 136us) server_negotiation.cc:407] Connection header received
0810 07:19:54.635393 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.635397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.635451 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.635540 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:54.635994 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.636843 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.638609 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.639217 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.639362 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.639418 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.639763 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.639865 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 07:19:54.640059 3992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.634909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:55695 (local address 127.1.125.193:45291)
0810 07:19:54.635140 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.635143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.635221 (+ 78us) server_negotiation.cc:407] Connection header received
0810 07:19:54.635393 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.635397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.635451 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.635540 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:54.635994 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.636824 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.638609 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.639268 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.639377 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.639418 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.639763 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.639922 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260810 07:19:54.640803 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896d5e1842864f56aa96987c02ac6ecb" candidate_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94254137e0f447b1bb0fcb4006a2ba93" is_pre_election: true
I20260810 07:19:54.641124 3442 raft_consensus.cc:2454] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a49ba49aa84fe0a6770a4f2bc255c4 in term 0.
I20260810 07:19:54.641156 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896d5e1842864f56aa96987c02ac6ecb" candidate_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817d027e6b1a425786bba9bdb35e8e53" is_pre_election: true
I20260810 07:19:54.641530 3888 raft_consensus.cc:2454] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a49ba49aa84fe0a6770a4f2bc255c4 in term 0.
I20260810 07:19:54.641651 3554 leader_election.cc:302] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a49ba49aa84fe0a6770a4f2bc255c4, 94254137e0f447b1bb0fcb4006a2ba93; no voters:
I20260810 07:19:54.641963 4005 raft_consensus.cc:2781] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:54.642066 4005 raft_consensus.cc:482] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:54.642098 4005 raft_consensus.cc:3037] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:54.643285 4005 raft_consensus.cc:504] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } }
I20260810 07:19:54.643651 4005 leader_election.cc:288] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [CANDIDATE]: Term 1 election: Requested vote from peers 817d027e6b1a425786bba9bdb35e8e53 (127.1.125.195:33975), 94254137e0f447b1bb0fcb4006a2ba93 (127.1.125.193:45291)
I20260810 07:19:54.644016 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896d5e1842864f56aa96987c02ac6ecb" candidate_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817d027e6b1a425786bba9bdb35e8e53"
I20260810 07:19:54.644068 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896d5e1842864f56aa96987c02ac6ecb" candidate_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94254137e0f447b1bb0fcb4006a2ba93"
I20260810 07:19:54.644173 3442 raft_consensus.cc:3037] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:54.644167 3888 raft_consensus.cc:3037] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:54.645237 3442 raft_consensus.cc:2454] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a49ba49aa84fe0a6770a4f2bc255c4 in term 1.
I20260810 07:19:54.645692 3554 leader_election.cc:302] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a49ba49aa84fe0a6770a4f2bc255c4, 94254137e0f447b1bb0fcb4006a2ba93; no voters:
I20260810 07:19:54.645906 4005 raft_consensus.cc:2781] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:54.646157 4005 raft_consensus.cc:686] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 1 LEADER]: Becoming Leader. State: Replica: 31a49ba49aa84fe0a6770a4f2bc255c4, State: Running, Role: LEADER
I20260810 07:19:54.646543 4005 consensus_queue.cc:237] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } }
I20260810 07:19:54.650511 3204 catalog_manager.cc:5495] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31a49ba49aa84fe0a6770a4f2bc255c4 (127.1.125.194). New cstate: current_term: 1 leader_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } health_report { overall_health: UNKNOWN } } }
I20260810 07:19:54.650897 3888 raft_consensus.cc:2454] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a49ba49aa84fe0a6770a4f2bc255c4 in term 1.
I20260810 07:19:54.674701 3991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.670749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56346 (local address 127.1.125.194:34951)
0810 07:19:54.670873 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.670877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.670910 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:19:54.671241 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.671244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.671309 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.671414 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:54.671701 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.672233 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.673196 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.673413 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.673498 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.673838 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:54.674329 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:54.674418 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.674505 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.674582 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:19:54.680859 3888 raft_consensus.cc:1261] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53 [term 1 FOLLOWER]: Refusing update from remote peer 31a49ba49aa84fe0a6770a4f2bc255c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:54.680891 3442 raft_consensus.cc:1261] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93 [term 1 FOLLOWER]: Refusing update from remote peer 31a49ba49aa84fe0a6770a4f2bc255c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:54.681643 4011 consensus_queue.cc:1035] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:54.681838 4005 consensus_queue.cc:1035] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:54.689330 4018 mvcc.cc:205] Tried to move back new op lower bound from 7316874832604729344 to 7316874832475095040. Current Snapshot: MvccSnapshot[applied={T|T < 7316874832604729344}]
I20260810 07:19:54.691308 4020 mvcc.cc:205] Tried to move back new op lower bound from 7316874832604729344 to 7316874832475095040. Current Snapshot: MvccSnapshot[applied={T|T < 7316874832604729344}]
I20260810 07:19:54.691606 4019 mvcc.cc:205] Tried to move back new op lower bound from 7316874832604729344 to 7316874832475095040. Current Snapshot: MvccSnapshot[applied={T|T < 7316874832604729344 or (T in {7316874832604729344})}]
I20260810 07:19:54.702311 3204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:54.702555 3204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:54.705214 3204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260: Sending DeleteTablet for 3 replicas of tablet 896d5e1842864f56aa96987c02ac6ecb
I20260810 07:19:54.706001 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 896d5e1842864f56aa96987c02ac6ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:54 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:51802
I20260810 07:19:54.706197 3630 tablet_service.cc:1514] Processing DeleteTablet for tablet 896d5e1842864f56aa96987c02ac6ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:54 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:56336
I20260810 07:19:54.706254 3409 tablet_service.cc:1514] Processing DeleteTablet for tablet 896d5e1842864f56aa96987c02ac6ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:54 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:43838
I20260810 07:19:54.706640 4035 tablet_replica.cc:330] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4: stopping tablet replica
I20260810 07:19:54.706921 4037 tablet_replica.cc:330] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93: stopping tablet replica
I20260810 07:19:54.706954 4035 raft_consensus.cc:2229] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:54.707312 4037 raft_consensus.cc:2229] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:54.707386 4035 raft_consensus.cc:2258] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:54.707587 3204 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:54.708055 3204 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:54.708078 4037 raft_consensus.cc:2258] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:54.708886 4035 ts_tablet_manager.cc:1905] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:54.709787 4037 ts_tablet_manager.cc:1905] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:54.712056 4035 ts_tablet_manager.cc:1918] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:54.712169 4035 log.cc:1196] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/896d5e1842864f56aa96987c02ac6ecb
I20260810 07:19:54.712563 4035 ts_tablet_manager.cc:1939] T 896d5e1842864f56aa96987c02ac6ecb P 31a49ba49aa84fe0a6770a4f2bc255c4: Deleting consensus metadata
I20260810 07:19:54.713104 4037 ts_tablet_manager.cc:1918] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:54.713276 4037 log.cc:1196] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/wal/wals/896d5e1842864f56aa96987c02ac6ecb
I20260810 07:19:54.713657 4037 ts_tablet_manager.cc:1939] T 896d5e1842864f56aa96987c02ac6ecb P 94254137e0f447b1bb0fcb4006a2ba93: Deleting consensus metadata
I20260810 07:19:54.713819 3409 tablet_service.cc:1467] Processing CreateTablet for tablet 9198ddd2f6ed4d6981785174ddefc50b (DEFAULT_TABLE table=test-table [id=46af0c6b2ed44057947a9c172b7989a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:54.714102 3409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9198ddd2f6ed4d6981785174ddefc50b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:54.714829 3180 catalog_manager.cc:4841] TS 94254137e0f447b1bb0fcb4006a2ba93 (127.1.125.193:45291): tablet 896d5e1842864f56aa96987c02ac6ecb (table test-table [id=b0a548cf308444ffa91582a920771f05]) successfully deleted
I20260810 07:19:54.715975 3996 tablet_bootstrap.cc:492] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93: Bootstrap starting.
I20260810 07:19:54.716281 3184 catalog_manager.cc:4841] TS 31a49ba49aa84fe0a6770a4f2bc255c4 (127.1.125.194:34951): tablet 896d5e1842864f56aa96987c02ac6ecb (table test-table [id=b0a548cf308444ffa91582a920771f05]) successfully deleted
I20260810 07:19:54.716725 3630 tablet_service.cc:1467] Processing CreateTablet for tablet 9198ddd2f6ed4d6981785174ddefc50b (DEFAULT_TABLE table=test-table [id=46af0c6b2ed44057947a9c172b7989a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:54.716967 3630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9198ddd2f6ed4d6981785174ddefc50b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:54.717124 3996 tablet_bootstrap.cc:654] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:54.717257 4034 tablet_replica.cc:330] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53: stopping tablet replica
I20260810 07:19:54.717598 4034 raft_consensus.cc:2229] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:54.717906 4034 raft_consensus.cc:2258] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:54.719144 3997 tablet_bootstrap.cc:492] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4: Bootstrap starting.
I20260810 07:19:54.719321 4034 ts_tablet_manager.cc:1905] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:54.720201 3997 tablet_bootstrap.cc:654] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:54.721074 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 9198ddd2f6ed4d6981785174ddefc50b (DEFAULT_TABLE table=test-table [id=46af0c6b2ed44057947a9c172b7989a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:54.721293 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9198ddd2f6ed4d6981785174ddefc50b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:54.722054 4034 ts_tablet_manager.cc:1918] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:54.722141 4034 log.cc:1196] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/896d5e1842864f56aa96987c02ac6ecb
I20260810 07:19:54.722446 4034 ts_tablet_manager.cc:1939] T 896d5e1842864f56aa96987c02ac6ecb P 817d027e6b1a425786bba9bdb35e8e53: Deleting consensus metadata
I20260810 07:19:54.723445 3993 tablet_bootstrap.cc:492] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53: Bootstrap starting.
I20260810 07:19:54.723692 3182 catalog_manager.cc:4841] TS 817d027e6b1a425786bba9bdb35e8e53 (127.1.125.195:33975): tablet 896d5e1842864f56aa96987c02ac6ecb (table test-table [id=b0a548cf308444ffa91582a920771f05]) successfully deleted
I20260810 07:19:54.724187 3997 tablet_bootstrap.cc:492] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4: No bootstrap required, opened a new log
I20260810 07:19:54.724290 3997 ts_tablet_manager.cc:1397] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 07:19:54.724625 3993 tablet_bootstrap.cc:654] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:54.724871 3997 raft_consensus.cc:348] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.724999 3997 raft_consensus.cc:374] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:54.725032 3997 raft_consensus.cc:729] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a49ba49aa84fe0a6770a4f2bc255c4, State: Initialized, Role: FOLLOWER
I20260810 07:19:54.725914 3996 tablet_bootstrap.cc:492] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93: No bootstrap required, opened a new log
I20260810 07:19:54.725997 3996 ts_tablet_manager.cc:1397] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20260810 07:19:54.726735 3996 raft_consensus.cc:348] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.726832 3996 raft_consensus.cc:374] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:54.726855 3996 raft_consensus.cc:729] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94254137e0f447b1bb0fcb4006a2ba93, State: Initialized, Role: FOLLOWER
I20260810 07:19:54.726997 3996 consensus_queue.cc:260] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.727190 3997 consensus_queue.cc:260] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.727505 3997 ts_tablet_manager.cc:1428] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260810 07:19:54.727800 3996 ts_tablet_manager.cc:1428] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260810 07:19:54.729658 3993 tablet_bootstrap.cc:492] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53: No bootstrap required, opened a new log
I20260810 07:19:54.729766 3993 ts_tablet_manager.cc:1397] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260810 07:19:54.730288 3993 raft_consensus.cc:348] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.730434 3993 raft_consensus.cc:374] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:54.730484 3993 raft_consensus.cc:729] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 817d027e6b1a425786bba9bdb35e8e53, State: Initialized, Role: FOLLOWER
I20260810 07:19:54.730646 3993 consensus_queue.cc:260] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.731400 3993 ts_tablet_manager.cc:1428] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 07:19:54.842669 4026 raft_consensus.cc:482] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:54.842844 4026 raft_consensus.cc:504] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.843467 4026 leader_election.cc:288] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94254137e0f447b1bb0fcb4006a2ba93 (127.1.125.193:45291), 817d027e6b1a425786bba9bdb35e8e53 (127.1.125.195:33975)
I20260810 07:19:54.844081 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9198ddd2f6ed4d6981785174ddefc50b" candidate_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817d027e6b1a425786bba9bdb35e8e53" is_pre_election: true
I20260810 07:19:54.844126 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9198ddd2f6ed4d6981785174ddefc50b" candidate_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94254137e0f447b1bb0fcb4006a2ba93" is_pre_election: true
I20260810 07:19:54.844254 3888 raft_consensus.cc:2454] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a49ba49aa84fe0a6770a4f2bc255c4 in term 0.
I20260810 07:19:54.844374 3442 raft_consensus.cc:2454] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a49ba49aa84fe0a6770a4f2bc255c4 in term 0.
I20260810 07:19:54.844655 3555 leader_election.cc:302] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a49ba49aa84fe0a6770a4f2bc255c4, 817d027e6b1a425786bba9bdb35e8e53; no voters:
I20260810 07:19:54.844841 4026 raft_consensus.cc:2781] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:54.844894 4026 raft_consensus.cc:482] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:54.844945 4026 raft_consensus.cc:3037] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:54.846202 4026 raft_consensus.cc:504] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.846560 4026 leader_election.cc:288] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [CANDIDATE]: Term 1 election: Requested vote from peers 94254137e0f447b1bb0fcb4006a2ba93 (127.1.125.193:45291), 817d027e6b1a425786bba9bdb35e8e53 (127.1.125.195:33975)
I20260810 07:19:54.846913 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9198ddd2f6ed4d6981785174ddefc50b" candidate_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817d027e6b1a425786bba9bdb35e8e53"
I20260810 07:19:54.846971 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9198ddd2f6ed4d6981785174ddefc50b" candidate_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94254137e0f447b1bb0fcb4006a2ba93"
I20260810 07:19:54.847028 3888 raft_consensus.cc:3037] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:54.847102 3442 raft_consensus.cc:3037] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:54.848052 3888 raft_consensus.cc:2454] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a49ba49aa84fe0a6770a4f2bc255c4 in term 1.
I20260810 07:19:54.848371 3555 leader_election.cc:302] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a49ba49aa84fe0a6770a4f2bc255c4, 817d027e6b1a425786bba9bdb35e8e53; no voters:
I20260810 07:19:54.848407 3442 raft_consensus.cc:2454] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a49ba49aa84fe0a6770a4f2bc255c4 in term 1.
I20260810 07:19:54.848582 4026 raft_consensus.cc:2781] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:54.848692 4026 raft_consensus.cc:686] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 1 LEADER]: Becoming Leader. State: Replica: 31a49ba49aa84fe0a6770a4f2bc255c4, State: Running, Role: LEADER
I20260810 07:19:54.848842 4026 consensus_queue.cc:237] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.850692 3204 catalog_manager.cc:5495] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31a49ba49aa84fe0a6770a4f2bc255c4 (127.1.125.194). New cstate: current_term: 1 leader_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } health_report { overall_health: UNKNOWN } } }
Aug 10 07:19:54 dist-test-slave-44d2 krb5kdc[3131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346393, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:54.906679 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.898168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39029 (local address 127.0.0.1:45320)
0810 07:19:54.898471 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:54.898489 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:54.898630 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:54.898904 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:54.898907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:54.899332 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:54.899428 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.899435 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.900271 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.900277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.900959 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:54.900969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.901435 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.901438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:54.901560 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.902082 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:54.902094 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:54.903625 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:54.905274 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.905284 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.905289 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.905524 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.905782 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:54.905784 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:54.905786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:54.905868 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:54.906206 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:54.906211 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:54.906320 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 07:19:54.906414 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 07:19:54.906527 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":96,"threads_started":1}
I20260810 07:19:54.906831 4054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:54.898240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45320 (local address 127.1.125.254:39029)
0810 07:19:54.898496 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:54.898507 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:54.898577 (+ 70us) server_negotiation.cc:407] Connection header received
0810 07:19:54.898739 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:54.898742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:54.898790 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:54.898883 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:54.899550 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.900130 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.901087 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:54.901324 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:54.901435 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:54.903760 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:54.903763 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:54.903765 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:54.903788 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:54.905173 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.905623 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.905626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.905627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.905701 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:54.905944 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:54.905947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:54.905948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:54.906113 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:54.906187 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:54.906502 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 07:19:54.906656 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":109,"threads_started":1}
I20260810 07:19:54.910838 3204 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:45320:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 07:19:54.911568 3204 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:54.916570 3630 tablet_service.cc:1467] Processing CreateTablet for tablet 98419906b4cb4fec8b5510053e6cd712 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3eca27b7edbd4b1a9534653ad156a122]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:19:54.916570 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 98419906b4cb4fec8b5510053e6cd712 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3eca27b7edbd4b1a9534653ad156a122]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:19:54.916538 3409 tablet_service.cc:1467] Processing CreateTablet for tablet 98419906b4cb4fec8b5510053e6cd712 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3eca27b7edbd4b1a9534653ad156a122]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:19:54.917016 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98419906b4cb4fec8b5510053e6cd712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:54.917016 3630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98419906b4cb4fec8b5510053e6cd712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:54.917210 3409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98419906b4cb4fec8b5510053e6cd712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:54.920013 3997 tablet_bootstrap.cc:492] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4: Bootstrap starting.
I20260810 07:19:54.920800 3996 tablet_bootstrap.cc:492] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93: Bootstrap starting.
I20260810 07:19:54.921030 3997 tablet_bootstrap.cc:654] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:54.921774 3996 tablet_bootstrap.cc:654] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:54.921916 3993 tablet_bootstrap.cc:492] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53: Bootstrap starting.
I20260810 07:19:54.922494 3997 tablet_bootstrap.cc:492] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4: No bootstrap required, opened a new log
I20260810 07:19:54.922575 3997 ts_tablet_manager.cc:1397] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:19:54.922904 3993 tablet_bootstrap.cc:654] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:54.922964 3997 raft_consensus.cc:348] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.923090 3997 raft_consensus.cc:374] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:54.923125 3997 raft_consensus.cc:729] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a49ba49aa84fe0a6770a4f2bc255c4, State: Initialized, Role: FOLLOWER
I20260810 07:19:54.923296 3996 tablet_bootstrap.cc:492] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93: No bootstrap required, opened a new log
I20260810 07:19:54.923290 3997 consensus_queue.cc:260] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.923370 3996 ts_tablet_manager.cc:1397] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260810 07:19:54.923609 4026 tablet_replica.cc:439] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } }
I20260810 07:19:54.923740 4026 tablet_replica.cc:442] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:54.923866 3996 raft_consensus.cc:348] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.923980 3996 raft_consensus.cc:374] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:54.924021 3996 raft_consensus.cc:729] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94254137e0f447b1bb0fcb4006a2ba93, State: Initialized, Role: FOLLOWER
I20260810 07:19:54.924153 3996 consensus_queue.cc:260] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.924417 3997 ts_tablet_manager.cc:1428] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:19:54.924427 4006 tablet_replica.cc:439] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } }
I20260810 07:19:54.924604 4006 tablet_replica.cc:442] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:54.924902 3996 ts_tablet_manager.cc:1428] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:54.925578 3993 tablet_bootstrap.cc:492] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53: No bootstrap required, opened a new log
I20260810 07:19:54.925688 3993 ts_tablet_manager.cc:1397] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260810 07:19:54.926195 3993 raft_consensus.cc:348] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.926326 3993 raft_consensus.cc:374] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:54.926386 3993 raft_consensus.cc:729] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 817d027e6b1a425786bba9bdb35e8e53, State: Initialized, Role: FOLLOWER
I20260810 07:19:54.926524 3993 consensus_queue.cc:260] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:54.927245 4004 tablet_replica.cc:439] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } }
I20260810 07:19:54.927466 4004 tablet_replica.cc:442] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:54.928110 3993 ts_tablet_manager.cc:1428] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:19:55.031986 4026 raft_consensus.cc:482] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:55.032135 4026 raft_consensus.cc:504] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:55.032697 4026 leader_election.cc:288] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94254137e0f447b1bb0fcb4006a2ba93 (127.1.125.193:45291), 817d027e6b1a425786bba9bdb35e8e53 (127.1.125.195:33975)
I20260810 07:19:55.033241 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98419906b4cb4fec8b5510053e6cd712" candidate_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94254137e0f447b1bb0fcb4006a2ba93" is_pre_election: true
I20260810 07:19:55.033241 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98419906b4cb4fec8b5510053e6cd712" candidate_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817d027e6b1a425786bba9bdb35e8e53" is_pre_election: true
I20260810 07:19:55.033417 3888 raft_consensus.cc:2454] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a49ba49aa84fe0a6770a4f2bc255c4 in term 0.
I20260810 07:19:55.033417 3442 raft_consensus.cc:2454] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a49ba49aa84fe0a6770a4f2bc255c4 in term 0.
I20260810 07:19:55.033754 3555 leader_election.cc:302] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a49ba49aa84fe0a6770a4f2bc255c4, 817d027e6b1a425786bba9bdb35e8e53; no voters:
I20260810 07:19:55.033952 4026 raft_consensus.cc:2781] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:55.034022 4026 raft_consensus.cc:482] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:55.034063 4026 raft_consensus.cc:3037] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:55.035060 4026 raft_consensus.cc:504] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:55.035386 4026 leader_election.cc:288] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [CANDIDATE]: Term 1 election: Requested vote from peers 94254137e0f447b1bb0fcb4006a2ba93 (127.1.125.193:45291), 817d027e6b1a425786bba9bdb35e8e53 (127.1.125.195:33975)
I20260810 07:19:55.035742 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98419906b4cb4fec8b5510053e6cd712" candidate_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817d027e6b1a425786bba9bdb35e8e53"
I20260810 07:19:55.035737 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98419906b4cb4fec8b5510053e6cd712" candidate_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94254137e0f447b1bb0fcb4006a2ba93"
I20260810 07:19:55.035864 3442 raft_consensus.cc:3037] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:55.035864 3888 raft_consensus.cc:3037] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:55.037055 3888 raft_consensus.cc:2454] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a49ba49aa84fe0a6770a4f2bc255c4 in term 1.
I20260810 07:19:55.037055 3442 raft_consensus.cc:2454] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a49ba49aa84fe0a6770a4f2bc255c4 in term 1.
I20260810 07:19:55.037365 3555 leader_election.cc:302] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a49ba49aa84fe0a6770a4f2bc255c4, 817d027e6b1a425786bba9bdb35e8e53; no voters:
I20260810 07:19:55.037586 4026 raft_consensus.cc:2781] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:55.037680 4026 raft_consensus.cc:686] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 1 LEADER]: Becoming Leader. State: Replica: 31a49ba49aa84fe0a6770a4f2bc255c4, State: Running, Role: LEADER
I20260810 07:19:55.037843 4026 consensus_queue.cc:237] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } }
I20260810 07:19:55.038594 4005 tablet_replica.cc:439] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4: TxnStatusTablet state changed. Reason: New leader 31a49ba49aa84fe0a6770a4f2bc255c4. Latest consensus state: current_term: 1 leader_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } }
I20260810 07:19:55.038703 4005 tablet_replica.cc:442] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:19:55.039072 4066 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 07:19:55.039361 4066 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 07:19:55.039924 3203 catalog_manager.cc:5495] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31a49ba49aa84fe0a6770a4f2bc255c4 (127.1.125.194). New cstate: current_term: 1 leader_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } health_report { overall_health: UNKNOWN } } }
I20260810 07:19:55.081274 3991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.077635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56362 (local address 127.1.125.194:34951)
0810 07:19:55.077757 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.077762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.077778 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:19:55.077942 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.077946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.078005 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.078094 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:55.078340 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.079118 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.079991 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.080371 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.080472 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.080810 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:55.080898 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:55.080964 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.081073 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.081130 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:19:55.081307 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.077502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:34951 (local address 127.0.0.1:56362)
0810 07:19:55.077694 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:55.077711 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:55.077819 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:55.078126 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:55.078129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:55.078135 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:19:55.078235 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.078240 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.079229 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.079232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.079892 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.079902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.080524 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.080527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.080619 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.080671 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:19:55.080982 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:19:55.080997 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.081047 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.081126 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 07:19:55.084234 3888 raft_consensus.cc:1261] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53 [term 1 FOLLOWER]: Refusing update from remote peer 31a49ba49aa84fe0a6770a4f2bc255c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:55.084383 3442 raft_consensus.cc:1261] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93 [term 1 FOLLOWER]: Refusing update from remote peer 31a49ba49aa84fe0a6770a4f2bc255c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:55.084726 4026 consensus_queue.cc:1035] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:55.084878 4005 consensus_queue.cc:1035] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:55.086376 4004 tablet_replica.cc:439] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53: TxnStatusTablet state changed. Reason: New leader 31a49ba49aa84fe0a6770a4f2bc255c4. Latest consensus state: current_term: 1 leader_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } }
I20260810 07:19:55.086387 4006 tablet_replica.cc:439] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93: TxnStatusTablet state changed. Reason: New leader 31a49ba49aa84fe0a6770a4f2bc255c4. Latest consensus state: current_term: 1 leader_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } }
I20260810 07:19:55.086493 4004 tablet_replica.cc:442] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:55.086503 4006 tablet_replica.cc:442] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:55.089309 4005 tablet_replica.cc:439] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } }
I20260810 07:19:55.089421 4005 tablet_replica.cc:442] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:19:55.090147 4004 tablet_replica.cc:439] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } }
I20260810 07:19:55.090256 4004 tablet_replica.cc:442] T 98419906b4cb4fec8b5510053e6cd712 P 817d027e6b1a425786bba9bdb35e8e53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:55.090587 4026 tablet_replica.cc:439] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } }
I20260810 07:19:55.090692 4026 tablet_replica.cc:442] T 98419906b4cb4fec8b5510053e6cd712 P 31a49ba49aa84fe0a6770a4f2bc255c4: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:19:55.093147 4006 tablet_replica.cc:439] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 } } peers { permanent_uuid: "31a49ba49aa84fe0a6770a4f2bc255c4" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34951 } } peers { permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 } } }
I20260810 07:19:55.093279 4006 tablet_replica.cc:442] T 98419906b4cb4fec8b5510053e6cd712 P 94254137e0f447b1bb0fcb4006a2ba93: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:55.105494 3991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.101669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:43697 (local address 127.1.125.194:34951)
0810 07:19:55.101787 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.101790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.102061 (+ 271us) server_negotiation.cc:407] Connection header received
0810 07:19:55.102226 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.102229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.102271 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.102335 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:55.102598 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.103188 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.104381 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.104668 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.104752 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.105056 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:55.105126 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:55.105181 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.105307 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.105350 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:19:55.105541 4079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.101587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:34951 (local address 127.1.125.194:43697)
0810 07:19:55.101982 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:55.101998 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:55.102125 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:55.102377 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:55.102380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:55.102388 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:19:55.102493 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.102499 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.103293 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.103297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.104268 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:55.104280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.104777 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.104781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:55.104871 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.104924 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:19:55.105209 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:19:55.105215 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:19:55.105282 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 07:19:55.105373 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":116,"threads_started":1}
I20260810 07:19:55.112437 3442 raft_consensus.cc:1261] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93 [term 1 FOLLOWER]: Refusing update from remote peer 31a49ba49aa84fe0a6770a4f2bc255c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:55.112607 3888 raft_consensus.cc:1261] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53 [term 1 FOLLOWER]: Refusing update from remote peer 31a49ba49aa84fe0a6770a4f2bc255c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:55.113005 4005 consensus_queue.cc:1035] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "817d027e6b1a425786bba9bdb35e8e53" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:55.113193 4082 consensus_queue.cc:1035] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94254137e0f447b1bb0fcb4006a2ba93" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 45291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:55.154335 3204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:55.154539 3204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:55.156742 3204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d21e92387112427b854a4d30fbfcd260: Sending DeleteTablet for 3 replicas of tablet 9198ddd2f6ed4d6981785174ddefc50b
I20260810 07:19:55.157300 3409 tablet_service.cc:1514] Processing DeleteTablet for tablet 9198ddd2f6ed4d6981785174ddefc50b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:55 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:43838
I20260810 07:19:55.157300 3628 tablet_service.cc:1514] Processing DeleteTablet for tablet 9198ddd2f6ed4d6981785174ddefc50b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:55 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:56336
I20260810 07:19:55.157491 4037 tablet_replica.cc:330] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93: stopping tablet replica
I20260810 07:19:55.157604 4035 tablet_replica.cc:330] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4: stopping tablet replica
I20260810 07:19:55.157608 4037 raft_consensus.cc:2229] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:55.157747 4035 raft_consensus.cc:2229] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:55.157755 4037 raft_consensus.cc:2258] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:55.157764 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 9198ddd2f6ed4d6981785174ddefc50b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:55 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:51802
I20260810 07:19:55.157963 4035 raft_consensus.cc:2258] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:55.157987 4034 tablet_replica.cc:330] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53: stopping tablet replica
I20260810 07:19:55.158123 4034 raft_consensus.cc:2229] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:55.158259 4034 raft_consensus.cc:2258] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:55.158957 4035 ts_tablet_manager.cc:1905] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:55.159646 4037 ts_tablet_manager.cc:1905] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:55.160308 4034 ts_tablet_manager.cc:1905] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:55.162377 4035 ts_tablet_manager.cc:1918] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:55.162469 4035 log.cc:1196] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/9198ddd2f6ed4d6981785174ddefc50b
I20260810 07:19:55.162693 4037 ts_tablet_manager.cc:1918] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:55.162745 4035 ts_tablet_manager.cc:1939] T 9198ddd2f6ed4d6981785174ddefc50b P 31a49ba49aa84fe0a6770a4f2bc255c4: Deleting consensus metadata
I20260810 07:19:55.162765 4037 log.cc:1196] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-0/wal/wals/9198ddd2f6ed4d6981785174ddefc50b
I20260810 07:19:55.163024 4037 ts_tablet_manager.cc:1939] T 9198ddd2f6ed4d6981785174ddefc50b P 94254137e0f447b1bb0fcb4006a2ba93: Deleting consensus metadata
I20260810 07:19:55.163390 4034 ts_tablet_manager.cc:1918] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:55.163465 4034 log.cc:1196] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/9198ddd2f6ed4d6981785174ddefc50b
I20260810 07:19:55.163437 3184 catalog_manager.cc:4841] TS 31a49ba49aa84fe0a6770a4f2bc255c4 (127.1.125.194:34951): tablet 9198ddd2f6ed4d6981785174ddefc50b (table test-table [id=46af0c6b2ed44057947a9c172b7989a9]) successfully deleted
I20260810 07:19:55.163715 3180 catalog_manager.cc:4841] TS 94254137e0f447b1bb0fcb4006a2ba93 (127.1.125.193:45291): tablet 9198ddd2f6ed4d6981785174ddefc50b (table test-table [id=46af0c6b2ed44057947a9c172b7989a9]) successfully deleted
I20260810 07:19:55.163975 4034 ts_tablet_manager.cc:1939] T 9198ddd2f6ed4d6981785174ddefc50b P 817d027e6b1a425786bba9bdb35e8e53: Deleting consensus metadata
I20260810 07:19:55.164773 3182 catalog_manager.cc:4841] TS 817d027e6b1a425786bba9bdb35e8e53 (127.1.125.195:33975): tablet 9198ddd2f6ed4d6981785174ddefc50b (table test-table [id=46af0c6b2ed44057947a9c172b7989a9]) successfully deleted
Aug 10 07:19:55 dist-test-slave-44d2 krb5kdc[3131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346394, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.125.193@KRBTEST.COM
I20260810 07:19:55.180117 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.164275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43844 (local address 127.1.125.193:45291)
0810 07:19:55.164884 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.164889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.168860 (+ 3971us) server_negotiation.cc:407] Connection header received
0810 07:19:55.169056 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.169060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.169122 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.169224 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:55.169995 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.170574 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.171753 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.171977 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.172056 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.176198 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.176214 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.176240 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.176267 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.178147 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.178611 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.178623 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.178625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.178710 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.179004 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.179007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.179009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.179297 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.179550 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.179746 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.179879 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":65,"threads_started":1}
W20260810 07:19:55.180670 3476 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43844
I20260810 07:19:55.190761 4054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:55.182515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45332 (local address 127.1.125.254:39029)
0810 07:19:55.182653 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:55.182657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:55.182673 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:19:55.182794 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:55.182797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:55.182858 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:55.182955 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:55.183586 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.184436 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.185506 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:55.185862 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:55.185973 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:55.187465 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:55.187469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:55.187472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:55.187503 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:55.189093 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.189498 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.189502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.189504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.189596 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:55.189878 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:55.189881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:55.189883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:55.190095 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:55.190223 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:55.190441 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 07:19:55.190585 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 07:19:55.191231 3204 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45332
I20260810 07:19:55.192621 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 3320
I20260810 07:19:55.210673 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 3529
I20260810 07:19:55.228906 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 3744
I20260810 07:19:55.237588 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 3152
2026-08-10T07:19:55Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3466 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:55 dist-test-slave-44d2 krb5kdc[4104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:55 dist-test-slave-44d2 krb5kdc[4104](info): set up 2 sockets
Aug 10 07:19:55 dist-test-slave-44d2 krb5kdc[4104](info): commencing operation
krb5kdc: starting...
W20260810 07:19:56.510360 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:19:56 dist-test-slave-44d2 krb5kdc[4104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:19:56.639462 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38045
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:38045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:56.754175 4122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:56.754429 4122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:56.754468 4122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:56.758482 4122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:56.758548 4122 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 07:19:56.758565 4122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:56.758612 4122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:56.758647 4122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:56.758667 4122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:56.763872 4122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:38045
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:56.765064 4122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:56.766083 4122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:56.775594 4130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:56.775826 4122 server_base.cc:1034] running on GCE node
W20260810 07:19:56.775604 4134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:56.775664 4131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:56.776666 4122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:56.777719 4122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:56.778888 4122 hybrid_clock.cc:648] HybridClock initialized: now 1786346396778857 us; error 43 us; skew 500 ppm
Aug 10 07:19:56 dist-test-slave-44d2 krb5kdc[4104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346396, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:56.786867 4122 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:19:56.787885 4122 webserver.cc:460] Webserver started at http://127.1.125.254:40591/ using document root <none> and password file <none>
I20260810 07:19:56.788533 4122 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:56.788630 4122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:56.788883 4122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:56.790970 4122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "a156e283e774438e9644e555d31c2a20"
format_stamp: "Formatted at 2026-08-10 07:19:56 on dist-test-slave-44d2"
server_key: "b1786c6d03edb3ec81a8734167c719ee"
server_key_iv: "3925d8f6003216cf980b919a0c93cfd2"
server_key_version: "encryptionkey@0"
I20260810 07:19:56.791556 4122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "a156e283e774438e9644e555d31c2a20"
format_stamp: "Formatted at 2026-08-10 07:19:56 on dist-test-slave-44d2"
server_key: "b1786c6d03edb3ec81a8734167c719ee"
server_key_iv: "3925d8f6003216cf980b919a0c93cfd2"
server_key_version: "encryptionkey@0"
I20260810 07:19:56.795372 4122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:19:56.797909 4143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:56.799116 4122 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:19:56.799260 4122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "a156e283e774438e9644e555d31c2a20"
format_stamp: "Formatted at 2026-08-10 07:19:56 on dist-test-slave-44d2"
server_key: "b1786c6d03edb3ec81a8734167c719ee"
server_key_iv: "3925d8f6003216cf980b919a0c93cfd2"
server_key_version: "encryptionkey@0"
I20260810 07:19:56.799371 4122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:56.818212 4122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:56.821216 4122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:56.821424 4122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:56.830186 4122 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:38045
I20260810 07:19:56.830212 4238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:38045 every 8 connection(s)
I20260810 07:19:56.831238 4122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:56.834070 4239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:56.835917 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 4122
I20260810 07:19:56.836043 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:19:56.836326 1527 external_mini_cluster.cc:1428] Setting key 9b52464729c799c6ab82596b4ded33c4
I20260810 07:19:56.840739 4239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20: Bootstrap starting.
I20260810 07:19:56.843046 4239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:56.843817 4239 log.cc:824] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20: Log is configured to *not* fsync() on all Append() calls
Aug 10 07:19:56 dist-test-slave-44d2 krb5kdc[4104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:56.846199 4239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20: No bootstrap required, opened a new log
I20260810 07:19:56.849421 4239 raft_consensus.cc:348] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a156e283e774438e9644e555d31c2a20" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38045 } }
I20260810 07:19:56.849699 4239 raft_consensus.cc:374] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:56.849785 4239 raft_consensus.cc:729] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a156e283e774438e9644e555d31c2a20, State: Initialized, Role: FOLLOWER
I20260810 07:19:56.850306 4239 consensus_queue.cc:260] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a156e283e774438e9644e555d31c2a20" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38045 } }
I20260810 07:19:56.850486 4239 raft_consensus.cc:388] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:56.850601 4239 raft_consensus.cc:482] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:56.850765 4239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:56.851670 4239 raft_consensus.cc:504] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a156e283e774438e9644e555d31c2a20" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38045 } }
I20260810 07:19:56.851755 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:56.837889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39922 (local address 127.1.125.254:38045)
0810 07:19:56.838353 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:56.838368 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:56.838406 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:19:56.839094 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:56.839112 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:56.839439 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:56.839791 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:56.840677 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:56.841718 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:56.842581 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:56.843051 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:56.843237 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:56.846030 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:56.846035 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:56.846048 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:56.846076 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:56.848536 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:56.849052 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:56.849057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:56.849062 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:56.849172 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:56.849508 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:56.849510 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:56.849511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:56.850030 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:56.850187 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:56.850529 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 07:19:56.851001 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":149,"threads_started":1}
I20260810 07:19:56.852068 4239 leader_election.cc:302] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a156e283e774438e9644e555d31c2a20; no voters:
I20260810 07:19:56.852451 4239 leader_election.cc:288] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:56.852577 4248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:56.852803 4248 raft_consensus.cc:686] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [term 1 LEADER]: Becoming Leader. State: Replica: a156e283e774438e9644e555d31c2a20, State: Running, Role: LEADER
I20260810 07:19:56.853184 4248 consensus_queue.cc:237] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a156e283e774438e9644e555d31c2a20" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38045 } }
I20260810 07:19:56.854835 4239 sys_catalog.cc:564] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:56.854959 4250 sys_catalog.cc:455] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a156e283e774438e9644e555d31c2a20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a156e283e774438e9644e555d31c2a20" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38045 } } }
I20260810 07:19:56.855101 4250 sys_catalog.cc:458] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:56.855428 4253 sys_catalog.cc:455] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a156e283e774438e9644e555d31c2a20. Latest consensus state: current_term: 1 leader_uuid: "a156e283e774438e9644e555d31c2a20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a156e283e774438e9644e555d31c2a20" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38045 } } }
I20260810 07:19:56.855518 4253 sys_catalog.cc:458] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:56.855500 4256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:56.858928 4256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:56.864836 4256 catalog_manager.cc:1347] Generated new cluster ID: 990decce54c44597b12b3df514b37a91
I20260810 07:19:56.864918 4256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:56.877359 4256 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:56.878273 4256 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:56.888775 4256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20: Generated new TSK 0
I20260810 07:19:56.889452 4256 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:19:56.924324 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38045
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:57.042022 4278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:57.042280 4278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:57.042327 4278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:57.046283 4278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:57.046335 4278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:57.046375 4278 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 07:19:57.046389 4278 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:19:57.046420 4278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:19:57.053009 4278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.125.254:38045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:57.054030 4278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:57.055246 4278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:57.066445 4289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:57.066545 4291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:57.066545 4288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:57.066758 4278 server_base.cc:1034] running on GCE node
I20260810 07:19:57.067286 4278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:57.067967 4278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:57.069162 4278 hybrid_clock.cc:648] HybridClock initialized: now 1786346397069144 us; error 31 us; skew 500 ppm
Aug 10 07:19:57 dist-test-slave-44d2 krb5kdc[4104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346397, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:57.072094 4278 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:19:57.073065 4278 webserver.cc:460] Webserver started at http://127.1.125.193:45425/ using document root <none> and password file <none>
I20260810 07:19:57.073670 4278 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:57.073767 4278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:57.074011 4278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:57.076067 4278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "be87462c243747a69094cf205190dc0d"
format_stamp: "Formatted at 2026-08-10 07:19:57 on dist-test-slave-44d2"
server_key: "f3736fd5d2b66471007f325c6922c6db"
server_key_iv: "1068294cddee808d7a854114184c3731"
server_key_version: "encryptionkey@0"
I20260810 07:19:57.076689 4278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "be87462c243747a69094cf205190dc0d"
format_stamp: "Formatted at 2026-08-10 07:19:57 on dist-test-slave-44d2"
server_key: "f3736fd5d2b66471007f325c6922c6db"
server_key_iv: "1068294cddee808d7a854114184c3731"
server_key_version: "encryptionkey@0"
I20260810 07:19:57.080413 4278 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:19:57.082952 4306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:57.084339 4278 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:19:57.084560 4278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "be87462c243747a69094cf205190dc0d"
format_stamp: "Formatted at 2026-08-10 07:19:57 on dist-test-slave-44d2"
server_key: "f3736fd5d2b66471007f325c6922c6db"
server_key_iv: "1068294cddee808d7a854114184c3731"
server_key_version: "encryptionkey@0"
I20260810 07:19:57.084682 4278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:57.095572 4278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:57.099130 4278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:57.099351 4278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:57.101045 4278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:57.101130 4278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:57.101208 4278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:57.101260 4278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:19:57 dist-test-slave-44d2 krb5kdc[4104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346397, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:57.113293 4278 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:32885
I20260810 07:19:57.113757 4488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:32885 every 8 connection(s)
I20260810 07:19:57.114439 4278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:57.116624 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.101822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:39135 (local address 127.1.125.254:38045)
0810 07:19:57.102005 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.102011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.103526 (+ 1515us) server_negotiation.cc:407] Connection header received
0810 07:19:57.105163 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.105167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.105225 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.105314 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:57.106837 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.107392 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.108875 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.109115 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.109241 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.112251 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:57.112255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:57.112258 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:57.112290 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:57.114218 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.114823 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.114827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.114829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.114995 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.115424 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.115427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.115429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.115633 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:57.115748 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.116276 (+ 528us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.116415 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 07:19:57.117388 4326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.102780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38045 (local address 127.1.125.193:39135)
0810 07:19:57.103341 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.103410 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.104878 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.105807 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.105818 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.106187 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:57.106675 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.106689 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.107534 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.107538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.108739 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.108752 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.109301 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.109306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.109489 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.110180 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:57.110197 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:57.112077 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:57.114344 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.114350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.114366 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.114705 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.115118 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.115121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.115123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.115312 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.115748 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:57.115769 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:57.116029 (+ 260us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.116228 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.116584 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":160,"threads_started":1}
I20260810 07:19:57.118685 4489 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38045
I20260810 07:19:57.118860 4489 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:57.119412 4489 heartbeater.cc:511] Master 127.1.125.254:38045 requested a full tablet report, sending...
I20260810 07:19:57.121114 4168 ts_manager.cc:194] Registered new tserver with Master: be87462c243747a69094cf205190dc0d (127.1.125.193:32885)
I20260810 07:19:57.121447 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 4278
I20260810 07:19:57.121564 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:57.121881 1527 external_mini_cluster.cc:1428] Setting key d95945fff89c4e5b2a5518764308ecf1
I20260810 07:19:57.122686 4168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:39135
I20260810 07:19:57.131248 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.121020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:34459 (local address 127.1.125.254:38045)
0810 07:19:57.121160 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.121165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.123546 (+ 2381us) server_negotiation.cc:407] Connection header received
0810 07:19:57.123666 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.123669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.123716 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.123801 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:57.124427 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.125191 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.126184 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.126563 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.126685 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.128043 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:57.128046 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:57.128048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:57.128071 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:57.129593 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.130036 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.130040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.130041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.130123 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.130410 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.130415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.130417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.130598 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:57.130709 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.130972 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.131088 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 07:19:57.131259 4499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.120950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38045 (local address 127.1.125.193:34459)
0810 07:19:57.123446 (+ 2496us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.123462 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.123568 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.123852 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.123857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.124110 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:57.124282 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.124293 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.125342 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.125347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.126057 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.126067 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.126693 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.126696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.126768 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.127309 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:57.127329 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:57.127894 (+ 565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:57.129715 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.129718 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.129720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.129926 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.130218 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.130221 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.130223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.130318 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.130699 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:57.130703 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:57.130806 (+ 103us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.130917 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.131080 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2402,"thread_start_us":100,"threads_started":1}
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:19:57.153549 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38045
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:57.274658 4504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:57.274930 4504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:57.275038 4504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:57.279186 4504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:57.279254 4504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:57.279330 4504 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 07:19:57.279371 4504 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:19:57.279452 4504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:19:57.285130 4504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.125.254:38045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:57.286233 4504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:57.287271 4504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:57.299494 4516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:57.299477 4512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:57.299474 4510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:57.299924 4504 server_base.cc:1034] running on GCE node
I20260810 07:19:57.300450 4504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:57.301393 4504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:57.302596 4504 hybrid_clock.cc:648] HybridClock initialized: now 1786346397302531 us; error 82 us; skew 500 ppm
Aug 10 07:19:57 dist-test-slave-44d2 krb5kdc[4104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346397, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:57.305629 4504 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:19:57.306702 4504 webserver.cc:460] Webserver started at http://127.1.125.194:46837/ using document root <none> and password file <none>
I20260810 07:19:57.307343 4504 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:57.307438 4504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:57.307691 4504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:57.309887 4504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "3353bfe8017145ae83ebe5bd0c10fcb0"
format_stamp: "Formatted at 2026-08-10 07:19:57 on dist-test-slave-44d2"
server_key: "d017a4149ab3e8685cba38c1900b9423"
server_key_iv: "4c89a02d905f076759713a19e628ed1b"
server_key_version: "encryptionkey@0"
I20260810 07:19:57.310632 4504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "3353bfe8017145ae83ebe5bd0c10fcb0"
format_stamp: "Formatted at 2026-08-10 07:19:57 on dist-test-slave-44d2"
server_key: "d017a4149ab3e8685cba38c1900b9423"
server_key_iv: "4c89a02d905f076759713a19e628ed1b"
server_key_version: "encryptionkey@0"
I20260810 07:19:57.314663 4504 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:19:57.317360 4525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:57.318672 4504 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:57.318825 4504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "3353bfe8017145ae83ebe5bd0c10fcb0"
format_stamp: "Formatted at 2026-08-10 07:19:57 on dist-test-slave-44d2"
server_key: "d017a4149ab3e8685cba38c1900b9423"
server_key_iv: "4c89a02d905f076759713a19e628ed1b"
server_key_version: "encryptionkey@0"
I20260810 07:19:57.318977 4504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:57.348183 4504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:57.352057 4504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:57.352278 4504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:57.353907 4504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:57.353991 4504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:57.354054 4504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:57.354112 4504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:19:57 dist-test-slave-44d2 krb5kdc[4104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346397, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:57.366003 4504 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:44947
I20260810 07:19:57.366187 4699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:44947 every 8 connection(s)
I20260810 07:19:57.367254 4504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:57.369489 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.354831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:38881 (local address 127.1.125.254:38045)
0810 07:19:57.355006 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.355012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.356250 (+ 1238us) server_negotiation.cc:407] Connection header received
0810 07:19:57.357553 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.357560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.357629 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.357739 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:57.359583 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.360533 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.361849 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.362159 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.362264 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.365256 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:57.365260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:57.365263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:57.365289 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:57.367086 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.367697 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.367702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.367703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.367804 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.368290 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.368294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.368296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.368550 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:57.368667 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.369172 (+ 505us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.369305 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 07:19:57.370324 4537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.355423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38045 (local address 127.1.125.194:38881)
0810 07:19:57.356072 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.356128 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.357314 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.358109 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.358119 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.358630 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:57.359414 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.359439 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.360660 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.360667 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.361723 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.361738 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.362318 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.362322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.362561 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.363166 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:57.363182 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:57.365081 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:57.367210 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.367214 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.367226 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.367574 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.367927 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.367931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.367936 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.368169 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.368681 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:57.368689 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:57.368926 (+ 237us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.369128 (+ 202us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.369449 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":166,"threads_started":1}
I20260810 07:19:57.371760 4700 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38045
I20260810 07:19:57.371937 4700 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:57.372161 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 4504
I20260810 07:19:57.372282 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:57.372625 4700 heartbeater.cc:511] Master 127.1.125.254:38045 requested a full tablet report, sending...
I20260810 07:19:57.372650 1527 external_mini_cluster.cc:1428] Setting key fa3d8e3eb099c242769012ebba21be09
I20260810 07:19:57.373772 4168 ts_manager.cc:194] Registered new tserver with Master: 3353bfe8017145ae83ebe5bd0c10fcb0 (127.1.125.194:44947)
I20260810 07:19:57.374580 4168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:38881
I20260810 07:19:57.382072 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.374188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:37489 (local address 127.1.125.254:38045)
0810 07:19:57.374299 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.374303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.374599 (+ 296us) server_negotiation.cc:407] Connection header received
0810 07:19:57.374737 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.374740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.374785 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.374836 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:57.375445 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.375941 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.377019 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.377281 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.377373 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.378732 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:57.378735 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:57.378737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:57.378757 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:57.380371 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.380833 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.380836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.380838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.380926 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.381210 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.381214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.381216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.381429 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:57.381540 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.381811 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.381932 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:19:57.382085 4706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.374229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38045 (local address 127.1.125.194:37489)
0810 07:19:57.374483 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.374497 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.374614 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.374974 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.374976 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.375202 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:57.375325 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.375331 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.376073 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.376076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.376830 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.376845 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.377407 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.377414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.377536 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.378089 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:57.378112 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:57.378605 (+ 493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:57.380514 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.380518 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.380520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.380727 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.381022 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.381026 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.381028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.381119 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.381531 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:57.381534 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:57.381639 (+ 105us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.381744 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.381894 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:19:57.403827 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38045
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:57.523213 4712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:57.523561 4712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:57.523643 4712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:57.527706 4712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:57.527771 4712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:57.527812 4712 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 07:19:57.527868 4712 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:19:57.527928 4712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:19:57.533454 4712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.125.254:38045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:57.534588 4712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:57.535696 4712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:57.545748 4722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:57.545768 4724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:57.545730 4721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:57.545814 4712 server_base.cc:1034] running on GCE node
I20260810 07:19:57.546445 4712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:57.547145 4712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:57.548347 4712 hybrid_clock.cc:648] HybridClock initialized: now 1786346397548298 us; error 59 us; skew 500 ppm
Aug 10 07:19:57 dist-test-slave-44d2 krb5kdc[4104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346397, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:57.551213 4712 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:19:57.552150 4712 webserver.cc:460] Webserver started at http://127.1.125.195:44819/ using document root <none> and password file <none>
I20260810 07:19:57.552834 4712 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:57.552930 4712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:57.553180 4712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:57.555326 4712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "b2909decbff94f0fbe9edcb0980353dd"
format_stamp: "Formatted at 2026-08-10 07:19:57 on dist-test-slave-44d2"
server_key: "eb3a0d9adc2a3e4e5442b6ddfe65cbef"
server_key_iv: "0048bbe0b54909994af3df4efbbeebb4"
server_key_version: "encryptionkey@0"
I20260810 07:19:57.555970 4712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "b2909decbff94f0fbe9edcb0980353dd"
format_stamp: "Formatted at 2026-08-10 07:19:57 on dist-test-slave-44d2"
server_key: "eb3a0d9adc2a3e4e5442b6ddfe65cbef"
server_key_iv: "0048bbe0b54909994af3df4efbbeebb4"
server_key_version: "encryptionkey@0"
I20260810 07:19:57.560016 4712 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:19:57.562577 4735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:57.563828 4712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:19:57.564006 4712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "b2909decbff94f0fbe9edcb0980353dd"
format_stamp: "Formatted at 2026-08-10 07:19:57 on dist-test-slave-44d2"
server_key: "eb3a0d9adc2a3e4e5442b6ddfe65cbef"
server_key_iv: "0048bbe0b54909994af3df4efbbeebb4"
server_key_version: "encryptionkey@0"
I20260810 07:19:57.564131 4712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:57.577927 4712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:57.580803 4712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:57.581007 4712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:57.582626 4712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:57.582715 4712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:57.582768 4712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:57.582798 4712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:19:57 dist-test-slave-44d2 krb5kdc[4104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346397, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:57.594765 4712 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:40565
I20260810 07:19:57.594857 4919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:40565 every 8 connection(s)
I20260810 07:19:57.595968 4712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:57.597361 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.583221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:36441 (local address 127.1.125.254:38045)
0810 07:19:57.583353 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.583356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.585089 (+ 1733us) server_negotiation.cc:407] Connection header received
0810 07:19:57.586240 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.586244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.586307 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.586414 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:57.587634 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.588184 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.589945 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.590248 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.590366 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.593039 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:57.593044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:57.593046 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:57.593073 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:57.594823 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.595438 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.595442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.595443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.595524 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.595929 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.595931 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.595933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.596210 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:57.596293 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.597055 (+ 762us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.597185 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:19:57.598127 4744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.584333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38045 (local address 127.1.125.195:36441)
0810 07:19:57.584862 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.584939 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.586024 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.586593 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.586600 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.586941 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:57.587472 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.587490 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.588319 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.588323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.589809 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.589820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.590419 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.590423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.590599 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.591227 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:57.591253 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:57.592849 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:57.594963 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.594968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.594983 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.595317 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.595625 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.595629 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.595631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.595828 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.596327 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:57.596335 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:57.596730 (+ 395us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.596978 (+ 248us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.597310 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":168,"threads_started":1}
I20260810 07:19:57.599444 4920 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38045
I20260810 07:19:57.599637 4920 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:57.600296 4920 heartbeater.cc:511] Master 127.1.125.254:38045 requested a full tablet report, sending...
I20260810 07:19:57.601541 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 4712
I20260810 07:19:57.601591 4169 ts_manager.cc:194] Registered new tserver with Master: b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565)
I20260810 07:19:57.601687 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:57.602092 1527 external_mini_cluster.cc:1428] Setting key c11027b0f60014647e689cf7d44fe1c5
I20260810 07:19:57.602344 4169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:36441
I20260810 07:19:57.604722 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:19:57.611011 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.602027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38045 (local address 127.1.125.195:38923)
0810 07:19:57.602441 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.602459 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.602595 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.603347 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.603351 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.603587 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:57.603757 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.603768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.604737 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.604741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.605397 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.605407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.605892 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.605897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.605999 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.606686 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:57.606700 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:57.607115 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:57.609327 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.609330 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.609332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.609549 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.609904 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.609906 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.609908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.609991 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.610499 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:57.610501 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:57.610596 (+ 95us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.610692 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.610812 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":138,"threads_started":1}
I20260810 07:19:57.611356 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.602575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:38923 (local address 127.1.125.254:38045)
0810 07:19:57.602984 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.602989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.603016 (+ 27us) server_negotiation.cc:407] Connection header received
0810 07:19:57.603077 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.603083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.603139 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.603219 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:57.603968 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.604602 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.605533 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.605762 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.605886 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.607312 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:57.607319 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:57.607324 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:57.607358 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:57.609199 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.609666 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.609673 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.609678 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.609783 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.610090 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.610096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.610101 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.610378 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:57.610507 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.610800 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.610959 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319}
Aug 10 07:19:57 dist-test-slave-44d2 krb5kdc[4104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:19:57 dist-test-slave-44d2 krb5kdc[4104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:19:57.634941 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.626792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.1.125.254:38045)
0810 07:19:57.626927 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.626930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.626943 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:19:57.626991 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.626994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.627033 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.627106 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:57.627848 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.628377 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.629237 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.629447 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.629531 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.631881 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:57.631884 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:57.631887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:57.631910 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:57.633527 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.633899 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.633902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.633903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.633975 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.634232 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.634234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.634235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.634396 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:57.634496 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.634666 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.634761 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:19:57.637948 4169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:57.641029 4169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:57.655510 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.650275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41560 (local address 127.1.125.195:40565)
0810 07:19:57.650735 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.650762 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.650786 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:19:57.650866 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.650872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.651104 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.651297 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:57.651467 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.652659 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.654024 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.654788 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.655017 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.655155 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.655208 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.655341 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":121,"threads_started":1}
I20260810 07:19:57.655511 4945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.649793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:40565 (local address 127.0.0.1:41560)
0810 07:19:57.650180 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.650196 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.650289 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.651272 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.651274 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.651294 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:57.651370 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.651376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.652779 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.652783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.653883 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.653891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.654951 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.654955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.655086 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.655108 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.655177 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.655343 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":30,"thread_start_us":104,"threads_started":1}
I20260810 07:19:57.655532 4944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.649455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44390 (local address 127.1.125.194:44947)
0810 07:19:57.649976 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.649988 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.650015 (+ 27us) server_negotiation.cc:407] Connection header received
0810 07:19:57.650244 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.650251 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.650466 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.650630 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:57.651166 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.652217 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.653836 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.654672 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.654783 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.654951 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.655270 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.655377 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":162,"threads_started":1}
I20260810 07:19:57.655833 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.650036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:32885 (local address 127.0.0.1:36402)
0810 07:19:57.650883 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.650896 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.650977 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.651702 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.651704 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.651747 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:57.651857 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.651863 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.653128 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.653132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.654187 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.654197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.655395 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.655399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.655484 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.655506 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.655574 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.655673 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":90,"threads_started":1}
I20260810 07:19:57.656299 4949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.650437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36402 (local address 127.1.125.193:32885)
0810 07:19:57.651248 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.651255 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.651277 (+ 22us) server_negotiation.cc:407] Connection header received
0810 07:19:57.651354 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.651359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.651556 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.651724 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:57.651979 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.653008 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.654318 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.655067 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.655171 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.655970 (+ 799us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.656036 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.656191 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":92,"threads_started":1}
I20260810 07:19:57.656524 4943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.649313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:44947 (local address 127.0.0.1:44390)
0810 07:19:57.649818 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.649865 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.650013 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.650836 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.650843 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.650865 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:57.651025 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.651033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.652321 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.652326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.653696 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.653707 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.654817 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.654821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.654941 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.655022 (+ 81us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.655147 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.656327 (+ 1180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":1065,"thread_start_us":73,"threads_started":1}
I20260810 07:19:57.659387 4812 tablet_service.cc:1467] Processing CreateTablet for tablet 400b7ce5ac834f6797cc5d472bcd9b8a (DEFAULT_TABLE table=test-table [id=1c902c7bbe854829b6ec9a5bbb871e15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:57.659384 4589 tablet_service.cc:1467] Processing CreateTablet for tablet 400b7ce5ac834f6797cc5d472bcd9b8a (DEFAULT_TABLE table=test-table [id=1c902c7bbe854829b6ec9a5bbb871e15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:57.660588 4589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 400b7ce5ac834f6797cc5d472bcd9b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:57.660588 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 400b7ce5ac834f6797cc5d472bcd9b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:57.662065 4385 tablet_service.cc:1467] Processing CreateTablet for tablet 400b7ce5ac834f6797cc5d472bcd9b8a (DEFAULT_TABLE table=test-table [id=1c902c7bbe854829b6ec9a5bbb871e15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:57.663029 4385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 400b7ce5ac834f6797cc5d472bcd9b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:57.669059 4950 tablet_bootstrap.cc:492] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0: Bootstrap starting.
I20260810 07:19:57.669845 4953 tablet_bootstrap.cc:492] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d: Bootstrap starting.
I20260810 07:19:57.670226 4952 tablet_bootstrap.cc:492] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd: Bootstrap starting.
I20260810 07:19:57.671738 4950 tablet_bootstrap.cc:654] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:57.672013 4953 tablet_bootstrap.cc:654] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:57.672428 4952 tablet_bootstrap.cc:654] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:57.672791 4950 log.cc:824] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:57.672861 4953 log.cc:824] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:57.673355 4952 log.cc:824] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:57.675037 4953 tablet_bootstrap.cc:492] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d: No bootstrap required, opened a new log
I20260810 07:19:57.675037 4950 tablet_bootstrap.cc:492] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0: No bootstrap required, opened a new log
I20260810 07:19:57.675050 4952 tablet_bootstrap.cc:492] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd: No bootstrap required, opened a new log
I20260810 07:19:57.675375 4952 ts_tablet_manager.cc:1397] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:19:57.675387 4950 ts_tablet_manager.cc:1397] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260810 07:19:57.675388 4953 ts_tablet_manager.cc:1397] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:19:57.678781 4952 raft_consensus.cc:348] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.679031 4952 raft_consensus.cc:374] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:57.679087 4952 raft_consensus.cc:729] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2909decbff94f0fbe9edcb0980353dd, State: Initialized, Role: FOLLOWER
I20260810 07:19:57.679535 4952 consensus_queue.cc:260] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.679723 4953 raft_consensus.cc:348] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.679723 4950 raft_consensus.cc:348] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.679997 4953 raft_consensus.cc:374] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:57.679997 4950 raft_consensus.cc:374] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:57.680068 4953 raft_consensus.cc:729] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be87462c243747a69094cf205190dc0d, State: Initialized, Role: FOLLOWER
I20260810 07:19:57.680079 4950 raft_consensus.cc:729] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3353bfe8017145ae83ebe5bd0c10fcb0, State: Initialized, Role: FOLLOWER
I20260810 07:19:57.680358 4920 heartbeater.cc:503] Master 127.1.125.254:38045 was elected leader, sending a full tablet report...
I20260810 07:19:57.680645 4952 ts_tablet_manager.cc:1428] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd: Time spent starting tablet: real 0.005s user 0.002s sys 0.005s
I20260810 07:19:57.680546 4953 consensus_queue.cc:260] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.680576 4950 consensus_queue.cc:260] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.681231 4700 heartbeater.cc:503] Master 127.1.125.254:38045 was elected leader, sending a full tablet report...
I20260810 07:19:57.681538 4953 ts_tablet_manager.cc:1428] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 07:19:57.681538 4950 ts_tablet_manager.cc:1428] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 07:19:57.681874 4489 heartbeater.cc:503] Master 127.1.125.254:38045 was elected leader, sending a full tablet report...
I20260810 07:19:57.786104 4966 raft_consensus.cc:482] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:57.786324 4966 raft_consensus.cc:504] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.787479 4966 leader_election.cc:288] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be87462c243747a69094cf205190dc0d (127.1.125.193:32885), b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565)
I20260810 07:19:57.792198 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.787874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:56497 (local address 127.1.125.195:40565)
0810 07:19:57.788020 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.788027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.788178 (+ 151us) server_negotiation.cc:407] Connection header received
0810 07:19:57.788336 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.788340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.788406 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.788528 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:57.788861 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.789727 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.791042 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.791656 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.791747 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.791789 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.792038 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.792098 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 07:19:57.792207 4537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.787803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:40565 (local address 127.1.125.194:56497)
0810 07:19:57.788064 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.788082 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.788205 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.788533 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.788536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.788562 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:57.788739 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.788749 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.789843 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.789846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.790904 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.790913 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.791775 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.791779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.791884 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.791910 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.791991 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.792059 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":3}
I20260810 07:19:57.792959 4949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.787853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:39473 (local address 127.1.125.193:32885)
0810 07:19:57.787994 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.787999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.788525 (+ 526us) server_negotiation.cc:407] Connection header received
0810 07:19:57.788668 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.788671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.788725 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.788812 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:57.789639 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.790348 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.791603 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.792219 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.792342 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.792398 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.792798 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.792868 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:19:57.792968 4969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.787803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:32885 (local address 127.1.125.194:39473)
0810 07:19:57.788364 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.788378 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.788552 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.788838 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.788843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.788863 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:57.789511 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.789522 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.790469 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.790472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.791461 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.791471 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.792512 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.792518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.792657 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.792682 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.792769 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.792834 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":74,"thread_start_us":186,"threads_started":1}
I20260810 07:19:57.792977 4849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "400b7ce5ac834f6797cc5d472bcd9b8a" candidate_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2909decbff94f0fbe9edcb0980353dd" is_pre_election: true
I20260810 07:19:57.793313 4849 raft_consensus.cc:2454] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3353bfe8017145ae83ebe5bd0c10fcb0 in term 0.
I20260810 07:19:57.793764 4414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "400b7ce5ac834f6797cc5d472bcd9b8a" candidate_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be87462c243747a69094cf205190dc0d" is_pre_election: true
I20260810 07:19:57.793834 4527 leader_election.cc:302] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3353bfe8017145ae83ebe5bd0c10fcb0, b2909decbff94f0fbe9edcb0980353dd; no voters:
I20260810 07:19:57.794081 4414 raft_consensus.cc:2454] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3353bfe8017145ae83ebe5bd0c10fcb0 in term 0.
I20260810 07:19:57.794081 4966 raft_consensus.cc:2781] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:57.794178 4966 raft_consensus.cc:482] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:57.794247 4966 raft_consensus.cc:3037] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:57.795337 4966 raft_consensus.cc:504] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.795635 4966 leader_election.cc:288] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [CANDIDATE]: Term 1 election: Requested vote from peers be87462c243747a69094cf205190dc0d (127.1.125.193:32885), b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565)
I20260810 07:19:57.796109 4414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "400b7ce5ac834f6797cc5d472bcd9b8a" candidate_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be87462c243747a69094cf205190dc0d"
I20260810 07:19:57.796103 4849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "400b7ce5ac834f6797cc5d472bcd9b8a" candidate_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2909decbff94f0fbe9edcb0980353dd"
I20260810 07:19:57.796257 4414 raft_consensus.cc:3037] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:57.796257 4849 raft_consensus.cc:3037] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:57.797802 4414 raft_consensus.cc:2454] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3353bfe8017145ae83ebe5bd0c10fcb0 in term 1.
I20260810 07:19:57.797802 4849 raft_consensus.cc:2454] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3353bfe8017145ae83ebe5bd0c10fcb0 in term 1.
I20260810 07:19:57.798167 4531 leader_election.cc:302] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3353bfe8017145ae83ebe5bd0c10fcb0, be87462c243747a69094cf205190dc0d; no voters:
I20260810 07:19:57.798346 4966 raft_consensus.cc:2781] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:57.798586 4966 raft_consensus.cc:686] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 LEADER]: Becoming Leader. State: Replica: 3353bfe8017145ae83ebe5bd0c10fcb0, State: Running, Role: LEADER
I20260810 07:19:57.798964 4966 consensus_queue.cc:237] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.802213 4168 catalog_manager.cc:5495] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3353bfe8017145ae83ebe5bd0c10fcb0 (127.1.125.194). New cstate: current_term: 1 leader_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } health_report { overall_health: UNKNOWN } } }
W20260810 07:19:57.847308 4921 tablet.cc:2369] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 07:19:57 dist-test-slave-44d2 krb5kdc[4104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346396, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
W20260810 07:19:57.866029 4490 tablet.cc:2369] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:57.866575 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.858190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.1.125.254:38045)
0810 07:19:57.858337 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.858342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.858562 (+ 220us) server_negotiation.cc:407] Connection header received
0810 07:19:57.858686 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.858689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.858729 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.858798 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:57.859434 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.859944 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.860741 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.860943 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.861024 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.863328 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:57.863332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:57.863334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:57.863355 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:57.864901 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.865311 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.865313 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.865315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.865385 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:57.865628 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:57.865630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:57.865631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:57.865788 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:57.865862 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.866260 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.866402 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":51}
I20260810 07:19:57.866611 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.858148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38045 (local address 127.0.0.1:39940)
0810 07:19:57.858470 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.858484 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.858609 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.858839 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.858841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.859223 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:57.859335 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.859342 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.860037 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.860039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.860639 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.860648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.861046 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.861048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.861123 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.861701 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:57.861713 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:57.863201 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:57.864998 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.865007 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.865011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.865229 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.865464 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:57.865466 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:57.865468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:57.865545 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:57.865894 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:57.865901 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:57.866050 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.866186 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.866348 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":138,"threads_started":1}
W20260810 07:19:57.868464 4701 tablet.cc:2369] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:57.870648 4168 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:39940:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 07:19:57.871343 4168 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:57.876456 4589 tablet_service.cc:1467] Processing CreateTablet for tablet 133240e1ec5d453083c2fc3f9e7e8edb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f95590b3309e48e19cdddf88c717e860]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:19:57.876411 4385 tablet_service.cc:1467] Processing CreateTablet for tablet 133240e1ec5d453083c2fc3f9e7e8edb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f95590b3309e48e19cdddf88c717e860]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:19:57.877214 4385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 133240e1ec5d453083c2fc3f9e7e8edb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:57.877214 4589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 133240e1ec5d453083c2fc3f9e7e8edb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:57.877107 4812 tablet_service.cc:1467] Processing CreateTablet for tablet 133240e1ec5d453083c2fc3f9e7e8edb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f95590b3309e48e19cdddf88c717e860]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:19:57.877746 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 133240e1ec5d453083c2fc3f9e7e8edb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:57.879823 4953 tablet_bootstrap.cc:492] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d: Bootstrap starting.
I20260810 07:19:57.880450 4950 tablet_bootstrap.cc:492] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0: Bootstrap starting.
I20260810 07:19:57.880658 4952 tablet_bootstrap.cc:492] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd: Bootstrap starting.
I20260810 07:19:57.880887 4953 tablet_bootstrap.cc:654] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:57.881569 4952 tablet_bootstrap.cc:654] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:57.881605 4950 tablet_bootstrap.cc:654] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:57.882359 4953 tablet_bootstrap.cc:492] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d: No bootstrap required, opened a new log
I20260810 07:19:57.882458 4953 ts_tablet_manager.cc:1397] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:19:57.882870 4953 raft_consensus.cc:348] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.882928 4950 tablet_bootstrap.cc:492] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0: No bootstrap required, opened a new log
I20260810 07:19:57.882957 4953 raft_consensus.cc:374] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:57.882979 4953 raft_consensus.cc:729] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be87462c243747a69094cf205190dc0d, State: Initialized, Role: FOLLOWER
I20260810 07:19:57.882968 4952 tablet_bootstrap.cc:492] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd: No bootstrap required, opened a new log
I20260810 07:19:57.883013 4950 ts_tablet_manager.cc:1397] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:19:57.883051 4952 ts_tablet_manager.cc:1397] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:57.883114 4953 consensus_queue.cc:260] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.883409 4953 ts_tablet_manager.cc:1428] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:19:57.883378 4961 tablet_replica.cc:439] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } } }
I20260810 07:19:57.883445 4952 raft_consensus.cc:348] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.883517 4961 tablet_replica.cc:442] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:57.883445 4950 raft_consensus.cc:348] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.883558 4950 raft_consensus.cc:374] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:57.883556 4952 raft_consensus.cc:374] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:57.883601 4950 raft_consensus.cc:729] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3353bfe8017145ae83ebe5bd0c10fcb0, State: Initialized, Role: FOLLOWER
I20260810 07:19:57.883601 4952 raft_consensus.cc:729] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2909decbff94f0fbe9edcb0980353dd, State: Initialized, Role: FOLLOWER
I20260810 07:19:57.883721 4952 consensus_queue.cc:260] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.883723 4950 consensus_queue.cc:260] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.883976 4950 ts_tablet_manager.cc:1428] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:19:57.883987 4952 ts_tablet_manager.cc:1428] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:57.884140 4966 tablet_replica.cc:439] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } } }
I20260810 07:19:57.884279 4966 tablet_replica.cc:442] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:57.884559 4957 tablet_replica.cc:439] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } } }
I20260810 07:19:57.884698 4957 tablet_replica.cc:442] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:57.902693 4961 raft_consensus.cc:482] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:57.902884 4961 raft_consensus.cc:504] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.904044 4961 leader_election.cc:288] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3353bfe8017145ae83ebe5bd0c10fcb0 (127.1.125.194:44947), b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565)
I20260810 07:19:57.908928 4944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.904368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:41657 (local address 127.1.125.194:44947)
0810 07:19:57.904573 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.904578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.904815 (+ 237us) server_negotiation.cc:407] Connection header received
0810 07:19:57.904911 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.904917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.904968 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.905048 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:57.905413 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.906187 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.907735 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.908290 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.908385 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.908427 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.908692 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.908792 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 07:19:57.908988 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.904288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:44947 (local address 127.1.125.193:41657)
0810 07:19:57.904718 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.904730 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.904826 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.905074 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.905077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.905094 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:57.905277 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.905285 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.906297 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.906301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.907607 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.907618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.908408 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.908412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.908540 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.908568 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.908653 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.908743 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":51,"thread_start_us":134,"threads_started":1}
I20260810 07:19:57.909206 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.904294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:40565 (local address 127.1.125.193:43121)
0810 07:19:57.904839 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.904854 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.904956 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.905217 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.905219 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.905235 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:57.905843 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.905855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.906751 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.906754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.908000 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.908011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.908771 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.908775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.908870 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.908894 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.908981 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.909054 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":28,"thread_start_us":171,"threads_started":1}
I20260810 07:19:57.909235 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.904378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:43121 (local address 127.1.125.195:40565)
0810 07:19:57.904563 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.904568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.904942 (+ 374us) server_negotiation.cc:407] Connection header received
0810 07:19:57.905064 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.905067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.905113 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.905191 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:57.905976 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.906610 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.908121 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.908651 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.908760 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.908811 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.909011 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.909096 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:19:57.909663 4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133240e1ec5d453083c2fc3f9e7e8edb" candidate_uuid: "be87462c243747a69094cf205190dc0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" is_pre_election: true
I20260810 07:19:57.909667 4849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133240e1ec5d453083c2fc3f9e7e8edb" candidate_uuid: "be87462c243747a69094cf205190dc0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2909decbff94f0fbe9edcb0980353dd" is_pre_election: true
I20260810 07:19:57.909835 4849 raft_consensus.cc:2454] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be87462c243747a69094cf205190dc0d in term 0.
I20260810 07:19:57.909929 4625 raft_consensus.cc:2454] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be87462c243747a69094cf205190dc0d in term 0.
I20260810 07:19:57.910246 4308 leader_election.cc:302] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2909decbff94f0fbe9edcb0980353dd, be87462c243747a69094cf205190dc0d; no voters:
I20260810 07:19:57.910558 4961 raft_consensus.cc:2781] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:57.910619 4961 raft_consensus.cc:482] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:57.910647 4961 raft_consensus.cc:3037] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:57.911705 4961 raft_consensus.cc:504] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.912103 4961 leader_election.cc:288] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [CANDIDATE]: Term 1 election: Requested vote from peers 3353bfe8017145ae83ebe5bd0c10fcb0 (127.1.125.194:44947), b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565)
I20260810 07:19:57.912449 4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133240e1ec5d453083c2fc3f9e7e8edb" candidate_uuid: "be87462c243747a69094cf205190dc0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0"
I20260810 07:19:57.912536 4849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133240e1ec5d453083c2fc3f9e7e8edb" candidate_uuid: "be87462c243747a69094cf205190dc0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2909decbff94f0fbe9edcb0980353dd"
I20260810 07:19:57.912611 4625 raft_consensus.cc:3037] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:57.912703 4849 raft_consensus.cc:3037] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:57.913616 4625 raft_consensus.cc:2454] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be87462c243747a69094cf205190dc0d in term 1.
I20260810 07:19:57.913765 4849 raft_consensus.cc:2454] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be87462c243747a69094cf205190dc0d in term 1.
I20260810 07:19:57.913954 4315 leader_election.cc:302] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3353bfe8017145ae83ebe5bd0c10fcb0, be87462c243747a69094cf205190dc0d; no voters:
I20260810 07:19:57.914181 4961 raft_consensus.cc:2781] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:57.914422 4961 raft_consensus.cc:686] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [term 1 LEADER]: Becoming Leader. State: Replica: be87462c243747a69094cf205190dc0d, State: Running, Role: LEADER
I20260810 07:19:57.914786 4961 consensus_queue.cc:237] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:57.917062 4988 tablet_replica.cc:439] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d: TxnStatusTablet state changed. Reason: New leader be87462c243747a69094cf205190dc0d. Latest consensus state: current_term: 1 leader_uuid: "be87462c243747a69094cf205190dc0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } } }
I20260810 07:19:57.917207 4988 tablet_replica.cc:442] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:19:57.917491 4991 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 07:19:57.917702 4991 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 07:19:57.918365 4169 catalog_manager.cc:5495] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d reported cstate change: term changed from 0 to 1, leader changed from <none> to be87462c243747a69094cf205190dc0d (127.1.125.193). New cstate: current_term: 1 leader_uuid: "be87462c243747a69094cf205190dc0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } health_report { overall_health: UNKNOWN } } }
I20260810 07:19:57.933143 4949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.929018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36412 (local address 127.1.125.193:32885)
0810 07:19:57.929141 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.929145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.929254 (+ 109us) server_negotiation.cc:407] Connection header received
0810 07:19:57.929444 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.929446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.929512 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.929594 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:57.929896 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.930401 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.931629 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.931841 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.931921 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.932285 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:57.932754 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:57.932842 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.932984 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.933038 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:19:57.933211 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.928910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:32885 (local address 127.0.0.1:36412)
0810 07:19:57.929175 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.929191 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.929312 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.929655 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.929659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.929669 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:19:57.929781 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.929788 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.930525 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.930529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.931500 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.931513 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.931965 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.931969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.932068 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.932131 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:19:57.932861 (+ 730us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:19:57.932879 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.932965 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.933062 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260810 07:19:57.938573 4625 raft_consensus.cc:1261] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Refusing update from remote peer be87462c243747a69094cf205190dc0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:57.938630 4849 raft_consensus.cc:1261] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Refusing update from remote peer be87462c243747a69094cf205190dc0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:57.939242 4961 consensus_queue.cc:1035] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:57.939436 4988 consensus_queue.cc:1035] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:57.942555 4966 tablet_replica.cc:439] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0: TxnStatusTablet state changed. Reason: New leader be87462c243747a69094cf205190dc0d. Latest consensus state: current_term: 1 leader_uuid: "be87462c243747a69094cf205190dc0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } } }
I20260810 07:19:57.942680 4966 tablet_replica.cc:442] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:57.944281 4957 tablet_replica.cc:439] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd: TxnStatusTablet state changed. Reason: New leader be87462c243747a69094cf205190dc0d. Latest consensus state: current_term: 1 leader_uuid: "be87462c243747a69094cf205190dc0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } } }
I20260810 07:19:57.944404 4957 tablet_replica.cc:442] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:57.946599 4988 tablet_replica.cc:439] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be87462c243747a69094cf205190dc0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } } }
I20260810 07:19:57.946717 4988 tablet_replica.cc:442] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:19:57.947312 4988 tablet_replica.cc:439] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be87462c243747a69094cf205190dc0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } } }
I20260810 07:19:57.947409 4988 tablet_replica.cc:442] T 133240e1ec5d453083c2fc3f9e7e8edb P be87462c243747a69094cf205190dc0d: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:19:57.948304 4957 tablet_replica.cc:439] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be87462c243747a69094cf205190dc0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } } }
I20260810 07:19:57.948411 4957 tablet_replica.cc:442] T 133240e1ec5d453083c2fc3f9e7e8edb P b2909decbff94f0fbe9edcb0980353dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:57.948392 5003 mvcc.cc:205] Tried to move back new op lower bound from 7316874845949526016 to 7316874845861466112. Current Snapshot: MvccSnapshot[applied={T|T < 7316874845949526016}]
I20260810 07:19:57.948681 5005 mvcc.cc:205] Tried to move back new op lower bound from 7316874845949526016 to 7316874845861466112. Current Snapshot: MvccSnapshot[applied={T|T < 7316874845949526016}]
I20260810 07:19:57.948678 4966 tablet_replica.cc:439] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be87462c243747a69094cf205190dc0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } } }
I20260810 07:19:57.948773 4966 tablet_replica.cc:442] T 133240e1ec5d453083c2fc3f9e7e8edb P 3353bfe8017145ae83ebe5bd0c10fcb0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:19:57.959278 4944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.955074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44406 (local address 127.1.125.194:44947)
0810 07:19:57.955205 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.955209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.955223 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:19:57.955392 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.955396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.955463 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.955579 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:57.955955 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.956785 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.957729 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.958069 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.958177 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.958427 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:57.958919 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:57.959038 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.959092 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.959146 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:19:57.971037 4949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.967249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:37845 (local address 127.1.125.193:32885)
0810 07:19:57.967393 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:57.967398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:57.967589 (+ 191us) server_negotiation.cc:407] Connection header received
0810 07:19:57.967721 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:57.967723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:57.967766 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:57.967850 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:57.968102 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.968721 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.969921 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.970141 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.970220 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.970614 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:57.970681 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:57.970730 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:57.970863 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 07:19:57.970911 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:19:57.971112 5020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:57.967141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:32885 (local address 127.1.125.194:37845)
0810 07:19:57.967495 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:57.967511 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:57.967622 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:57.967870 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:57.967874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:57.967882 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:19:57.968004 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.968010 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.968830 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.968834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.969807 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:57.969819 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:57.970275 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:57.970279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:57.970376 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:57.970431 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:19:57.970765 (+ 334us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:19:57.970772 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 07:19:57.970844 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 07:19:57.970927 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":157,"threads_started":1}
I20260810 07:19:57.979259 4846 raft_consensus.cc:1261] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Refusing update from remote peer 3353bfe8017145ae83ebe5bd0c10fcb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:57.979655 4414 raft_consensus.cc:1261] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Refusing update from remote peer 3353bfe8017145ae83ebe5bd0c10fcb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:57.980014 4966 consensus_queue.cc:1035] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:57.980262 4970 consensus_queue.cc:1035] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:57.984643 5002 mvcc.cc:205] Tried to move back new op lower bound from 7316874846118739968 to 7316874845386928128. Current Snapshot: MvccSnapshot[applied={T|T < 7316874846118739968}]
I20260810 07:19:58.005300 5032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.000963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:44947 (local address 127.1.125.193:53345)
0810 07:19:58.001604 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.001617 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:58.001711 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:58.001928 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:58.001930 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:58.001935 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:19:58.002050 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.002054 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.003091 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.003094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.004009 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.004020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.004637 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.004640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.004707 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.004750 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:19:58.005048 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:19:58.005053 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:19:58.005104 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 07:19:58.005166 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":132,"threads_started":1}
I20260810 07:19:58.005296 4944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.001031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:53345 (local address 127.1.125.194:44947)
0810 07:19:58.001166 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:58.001173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:58.001696 (+ 523us) server_negotiation.cc:407] Connection header received
0810 07:19:58.001787 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:58.001790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:58.001842 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:58.001927 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:58.002215 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.002935 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.004130 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.004476 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.004621 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.004877 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:58.004961 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:58.005034 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:58.005114 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 07:19:58.005167 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:19:58.030009 4169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:58.030205 4169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:58.033046 4169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20: Sending DeleteTablet for 3 replicas of tablet 400b7ce5ac834f6797cc5d472bcd9b8a
I20260810 07:19:58.033932 4589 tablet_service.cc:1514] Processing DeleteTablet for tablet 400b7ce5ac834f6797cc5d472bcd9b8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:58 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:44390
I20260810 07:19:58.034199 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 400b7ce5ac834f6797cc5d472bcd9b8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:58 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:41560
I20260810 07:19:58.034834 4385 tablet_service.cc:1514] Processing DeleteTablet for tablet 400b7ce5ac834f6797cc5d472bcd9b8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:58 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:36402
I20260810 07:19:58.035084 4169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:58.035490 4169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:58.035746 5037 tablet_replica.cc:330] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd: stopping tablet replica
I20260810 07:19:58.035912 5038 tablet_replica.cc:330] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d: stopping tablet replica
I20260810 07:19:58.036302 5037 raft_consensus.cc:2229] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:58.036690 5038 raft_consensus.cc:2229] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:58.037164 5035 tablet_replica.cc:330] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0: stopping tablet replica
I20260810 07:19:58.037425 5035 raft_consensus.cc:2229] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:58.037762 5035 raft_consensus.cc:2258] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:58.038913 5038 raft_consensus.cc:2258] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:58.039042 5037 raft_consensus.cc:2258] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:58.039227 5035 ts_tablet_manager.cc:1905] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:58.040802 4589 tablet_service.cc:1467] Processing CreateTablet for tablet 6237180b9b834820abe321d9bc38f696 (DEFAULT_TABLE table=test-table [id=116aeee4de6d415e9f18e3a7471669a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:58.040810 4385 tablet_service.cc:1467] Processing CreateTablet for tablet 6237180b9b834820abe321d9bc38f696 (DEFAULT_TABLE table=test-table [id=116aeee4de6d415e9f18e3a7471669a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:58.041067 4589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6237180b9b834820abe321d9bc38f696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:58.041086 4385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6237180b9b834820abe321d9bc38f696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:58.041044 4812 tablet_service.cc:1467] Processing CreateTablet for tablet 6237180b9b834820abe321d9bc38f696 (DEFAULT_TABLE table=test-table [id=116aeee4de6d415e9f18e3a7471669a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:58.041278 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6237180b9b834820abe321d9bc38f696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:58.043654 4950 tablet_bootstrap.cc:492] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0: Bootstrap starting.
I20260810 07:19:58.043680 4952 tablet_bootstrap.cc:492] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd: Bootstrap starting.
I20260810 07:19:58.043853 4953 tablet_bootstrap.cc:492] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d: Bootstrap starting.
I20260810 07:19:58.044705 4950 tablet_bootstrap.cc:654] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:58.044729 4952 tablet_bootstrap.cc:654] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:58.044827 4953 tablet_bootstrap.cc:654] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:58.046464 5038 ts_tablet_manager.cc:1905] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:58.047248 5037 ts_tablet_manager.cc:1905] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:58.050827 5035 ts_tablet_manager.cc:1918] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:58.050935 5035 log.cc:1196] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/400b7ce5ac834f6797cc5d472bcd9b8a
I20260810 07:19:58.051283 5035 ts_tablet_manager.cc:1939] T 400b7ce5ac834f6797cc5d472bcd9b8a P 3353bfe8017145ae83ebe5bd0c10fcb0: Deleting consensus metadata
I20260810 07:19:58.052006 5037 ts_tablet_manager.cc:1918] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:58.052656 5037 log.cc:1196] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/400b7ce5ac834f6797cc5d472bcd9b8a
I20260810 07:19:58.053172 4950 tablet_bootstrap.cc:492] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0: No bootstrap required, opened a new log
I20260810 07:19:58.053284 4950 ts_tablet_manager.cc:1397] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260810 07:19:58.053633 5037 ts_tablet_manager.cc:1939] T 400b7ce5ac834f6797cc5d472bcd9b8a P b2909decbff94f0fbe9edcb0980353dd: Deleting consensus metadata
I20260810 07:19:58.053833 4950 raft_consensus.cc:348] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:58.054003 4950 raft_consensus.cc:374] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:58.054049 4950 raft_consensus.cc:729] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3353bfe8017145ae83ebe5bd0c10fcb0, State: Initialized, Role: FOLLOWER
I20260810 07:19:58.054185 4950 consensus_queue.cc:260] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:58.054536 4950 ts_tablet_manager.cc:1428] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:19:58.054826 4150 catalog_manager.cc:4841] TS 3353bfe8017145ae83ebe5bd0c10fcb0 (127.1.125.194:44947): tablet 400b7ce5ac834f6797cc5d472bcd9b8a (table test-table [id=1c902c7bbe854829b6ec9a5bbb871e15]) successfully deleted
I20260810 07:19:58.055379 4952 tablet_bootstrap.cc:492] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd: No bootstrap required, opened a new log
I20260810 07:19:58.055488 4952 ts_tablet_manager.cc:1397] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260810 07:19:58.055913 4953 tablet_bootstrap.cc:492] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d: No bootstrap required, opened a new log
I20260810 07:19:58.056026 4953 ts_tablet_manager.cc:1397] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260810 07:19:58.056041 4952 raft_consensus.cc:348] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:58.056166 4952 raft_consensus.cc:374] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:58.056213 4952 raft_consensus.cc:729] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2909decbff94f0fbe9edcb0980353dd, State: Initialized, Role: FOLLOWER
I20260810 07:19:58.056351 4952 consensus_queue.cc:260] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:58.056660 4952 ts_tablet_manager.cc:1428] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:58.056632 4953 raft_consensus.cc:348] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:58.056793 4953 raft_consensus.cc:374] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:58.056864 4953 raft_consensus.cc:729] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be87462c243747a69094cf205190dc0d, State: Initialized, Role: FOLLOWER
I20260810 07:19:58.057026 4953 consensus_queue.cc:260] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:58.057338 4953 ts_tablet_manager.cc:1428] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260810 07:19:58.057381 4146 catalog_manager.cc:4841] TS b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565): tablet 400b7ce5ac834f6797cc5d472bcd9b8a (table test-table [id=1c902c7bbe854829b6ec9a5bbb871e15]) successfully deleted
I20260810 07:19:58.060782 5038 ts_tablet_manager.cc:1918] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:19:58.060917 5038 log.cc:1196] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/wal/wals/400b7ce5ac834f6797cc5d472bcd9b8a
I20260810 07:19:58.061345 5038 ts_tablet_manager.cc:1939] T 400b7ce5ac834f6797cc5d472bcd9b8a P be87462c243747a69094cf205190dc0d: Deleting consensus metadata
I20260810 07:19:58.062611 4150 catalog_manager.cc:4841] TS be87462c243747a69094cf205190dc0d (127.1.125.193:32885): tablet 400b7ce5ac834f6797cc5d472bcd9b8a (table test-table [id=1c902c7bbe854829b6ec9a5bbb871e15]) successfully deleted
I20260810 07:19:58.396082 5029 raft_consensus.cc:482] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:58.396238 5029 raft_consensus.cc:504] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:58.396822 5029 leader_election.cc:288] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3353bfe8017145ae83ebe5bd0c10fcb0 (127.1.125.194:44947), b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565)
I20260810 07:19:58.397331 4846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6237180b9b834820abe321d9bc38f696" candidate_uuid: "be87462c243747a69094cf205190dc0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2909decbff94f0fbe9edcb0980353dd" is_pre_election: true
I20260810 07:19:58.397361 4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6237180b9b834820abe321d9bc38f696" candidate_uuid: "be87462c243747a69094cf205190dc0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" is_pre_election: true
I20260810 07:19:58.397513 4625 raft_consensus.cc:2454] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be87462c243747a69094cf205190dc0d in term 0.
I20260810 07:19:58.397529 4846 raft_consensus.cc:2454] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be87462c243747a69094cf205190dc0d in term 0.
I20260810 07:19:58.397840 4315 leader_election.cc:302] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3353bfe8017145ae83ebe5bd0c10fcb0, be87462c243747a69094cf205190dc0d; no voters:
I20260810 07:19:58.398038 5029 raft_consensus.cc:2781] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:58.398111 5029 raft_consensus.cc:482] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:58.398164 5029 raft_consensus.cc:3037] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:58.399322 5029 raft_consensus.cc:504] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:58.399691 5029 leader_election.cc:288] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [CANDIDATE]: Term 1 election: Requested vote from peers 3353bfe8017145ae83ebe5bd0c10fcb0 (127.1.125.194:44947), b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565)
I20260810 07:19:58.400034 4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6237180b9b834820abe321d9bc38f696" candidate_uuid: "be87462c243747a69094cf205190dc0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0"
I20260810 07:19:58.400115 4846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6237180b9b834820abe321d9bc38f696" candidate_uuid: "be87462c243747a69094cf205190dc0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2909decbff94f0fbe9edcb0980353dd"
I20260810 07:19:58.400175 4625 raft_consensus.cc:3037] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:58.400257 4846 raft_consensus.cc:3037] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:58.401240 4625 raft_consensus.cc:2454] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be87462c243747a69094cf205190dc0d in term 1.
I20260810 07:19:58.401294 4846 raft_consensus.cc:2454] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be87462c243747a69094cf205190dc0d in term 1.
I20260810 07:19:58.401553 4315 leader_election.cc:302] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3353bfe8017145ae83ebe5bd0c10fcb0, be87462c243747a69094cf205190dc0d; no voters:
I20260810 07:19:58.401757 5029 raft_consensus.cc:2781] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:58.401859 5029 raft_consensus.cc:686] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [term 1 LEADER]: Becoming Leader. State: Replica: be87462c243747a69094cf205190dc0d, State: Running, Role: LEADER
I20260810 07:19:58.402029 5029 consensus_queue.cc:237] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:19:58.403848 4169 catalog_manager.cc:5495] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d reported cstate change: term changed from 0 to 1, leader changed from <none> to be87462c243747a69094cf205190dc0d (127.1.125.193). New cstate: current_term: 1 leader_uuid: "be87462c243747a69094cf205190dc0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } health_report { overall_health: UNKNOWN } } }
I20260810 07:19:58.420997 4949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.418136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36424 (local address 127.1.125.193:32885)
0810 07:19:58.418256 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:58.418260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:58.418272 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:19:58.418387 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:58.418389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:58.418458 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:58.418536 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:58.418763 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.419314 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.420091 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.420296 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.420373 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.420645 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:58.420711 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:58.420764 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:58.420846 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 07:19:58.420888 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:19:58.429889 4949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.426885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:38275 (local address 127.1.125.193:32885)
0810 07:19:58.427052 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:58.427058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:58.427075 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:19:58.427183 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:58.427186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:58.427254 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:58.427323 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:58.427551 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.428029 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.428902 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.429111 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.429216 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.429430 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:58.429503 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:58.429581 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:58.429691 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 07:19:58.429741 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":29}
I20260810 07:19:58.429889 5032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:58.426818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:32885 (local address 127.1.125.193:38275)
0810 07:19:58.426972 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:58.426991 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:58.427086 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:58.427330 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:58.427333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:58.427339 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:19:58.427450 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.427456 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.428133 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.428139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.428789 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:58.428798 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:58.429214 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:58.429218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:58.429299 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:58.429319 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:19:58.429583 (+ 264us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:19:58.429591 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:19:58.429680 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 07:19:58.429759 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":17}
I20260810 07:19:58.435846 4625 raft_consensus.cc:1261] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Refusing update from remote peer be87462c243747a69094cf205190dc0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:58.435988 4846 raft_consensus.cc:1261] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Refusing update from remote peer be87462c243747a69094cf205190dc0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:58.436262 5029 consensus_queue.cc:1035] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:58.436432 4961 consensus_queue.cc:1035] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:19:58.450693 5002 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 07:20:00.008677 4169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:00.008854 4169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:00.011602 4169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20: Sending DeleteTablet for 3 replicas of tablet 6237180b9b834820abe321d9bc38f696
I20260810 07:20:00.012236 4385 tablet_service.cc:1514] Processing DeleteTablet for tablet 6237180b9b834820abe321d9bc38f696 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:00 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:36402
I20260810 07:20:00.012359 4589 tablet_service.cc:1514] Processing DeleteTablet for tablet 6237180b9b834820abe321d9bc38f696 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:00 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:44390
I20260810 07:20:00.012637 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 6237180b9b834820abe321d9bc38f696 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:00 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:41560
I20260810 07:20:00.013073 5088 tablet_replica.cc:330] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd: stopping tablet replica
I20260810 07:20:00.013255 5088 raft_consensus.cc:2229] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:00.013394 5088 raft_consensus.cc:2258] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:00.014068 5088 ts_tablet_manager.cc:1905] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:00.016287 5088 ts_tablet_manager.cc:1918] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:20:00.016392 5088 log.cc:1196] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/6237180b9b834820abe321d9bc38f696
I20260810 07:20:00.016693 5085 tablet_replica.cc:330] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d: stopping tablet replica
I20260810 07:20:00.016929 5087 tablet_replica.cc:330] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0: stopping tablet replica
I20260810 07:20:00.017131 5087 raft_consensus.cc:2229] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:00.017204 5088 ts_tablet_manager.cc:1939] T 6237180b9b834820abe321d9bc38f696 P b2909decbff94f0fbe9edcb0980353dd: Deleting consensus metadata
I20260810 07:20:00.017292 5087 raft_consensus.cc:2258] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:00.017522 5085 raft_consensus.cc:2229] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:00.017891 5085 raft_consensus.cc:2258] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:00.017913 4146 catalog_manager.cc:4841] TS b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565): tablet 6237180b9b834820abe321d9bc38f696 (table test-table [id=116aeee4de6d415e9f18e3a7471669a4]) successfully deleted
I20260810 07:20:00.018260 5087 ts_tablet_manager.cc:1905] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:00.018684 5085 ts_tablet_manager.cc:1905] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:00.021348 5085 ts_tablet_manager.cc:1918] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:20:00.021440 5085 log.cc:1196] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/wal/wals/6237180b9b834820abe321d9bc38f696
I20260810 07:20:00.021620 5087 ts_tablet_manager.cc:1918] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:20:00.021718 5087 log.cc:1196] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/6237180b9b834820abe321d9bc38f696
I20260810 07:20:00.021767 5085 ts_tablet_manager.cc:1939] T 6237180b9b834820abe321d9bc38f696 P be87462c243747a69094cf205190dc0d: Deleting consensus metadata
I20260810 07:20:00.022014 5087 ts_tablet_manager.cc:1939] T 6237180b9b834820abe321d9bc38f696 P 3353bfe8017145ae83ebe5bd0c10fcb0: Deleting consensus metadata
I20260810 07:20:00.022575 4150 catalog_manager.cc:4841] TS be87462c243747a69094cf205190dc0d (127.1.125.193:32885): tablet 6237180b9b834820abe321d9bc38f696 (table test-table [id=116aeee4de6d415e9f18e3a7471669a4]) successfully deleted
I20260810 07:20:00.022794 4150 catalog_manager.cc:4841] TS 3353bfe8017145ae83ebe5bd0c10fcb0 (127.1.125.194:44947): tablet 6237180b9b834820abe321d9bc38f696 (table test-table [id=116aeee4de6d415e9f18e3a7471669a4]) successfully deleted
Aug 10 07:20:00 dist-test-slave-44d2 krb5kdc[4104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:20:00 dist-test-slave-44d2 krb5kdc[4104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346400, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:00.043814 5097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.034774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39944 (local address 127.1.125.254:38045)
0810 07:20:00.035064 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.035071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.035093 (+ 22us) server_negotiation.cc:407] Connection header received
0810 07:20:00.035216 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.035219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.035265 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.035348 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:00.035997 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.036774 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.037698 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.038007 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.038101 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.040771 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:00.040775 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:00.040778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:00.040803 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:00.042337 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.042725 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.042727 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.042729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.042811 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:00.043074 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:00.043077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:00.043078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:00.043259 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:00.043366 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.043566 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.043672 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":129,"threads_started":1}
I20260810 07:20:00.046545 4169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:20:00.047000 4169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:00.051728 4385 tablet_service.cc:1467] Processing CreateTablet for tablet d412e5546b1b4a9f89b78f898df41f2b (DEFAULT_TABLE table=test-table [id=8bd73ac2bbc94270b4ea84f0155a948f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:00.051785 4589 tablet_service.cc:1467] Processing CreateTablet for tablet d412e5546b1b4a9f89b78f898df41f2b (DEFAULT_TABLE table=test-table [id=8bd73ac2bbc94270b4ea84f0155a948f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:00.052023 4385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d412e5546b1b4a9f89b78f898df41f2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:00.052098 4589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d412e5546b1b4a9f89b78f898df41f2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:00.052114 4812 tablet_service.cc:1467] Processing CreateTablet for tablet d412e5546b1b4a9f89b78f898df41f2b (DEFAULT_TABLE table=test-table [id=8bd73ac2bbc94270b4ea84f0155a948f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:00.052393 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d412e5546b1b4a9f89b78f898df41f2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:00.054142 5099 tablet_bootstrap.cc:492] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d: Bootstrap starting.
I20260810 07:20:00.055087 5099 tablet_bootstrap.cc:654] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:00.055753 5100 tablet_bootstrap.cc:492] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0: Bootstrap starting.
I20260810 07:20:00.055827 5101 tablet_bootstrap.cc:492] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd: Bootstrap starting.
I20260810 07:20:00.056381 5099 tablet_bootstrap.cc:492] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d: No bootstrap required, opened a new log
I20260810 07:20:00.056489 5099 ts_tablet_manager.cc:1397] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:00.056675 5100 tablet_bootstrap.cc:654] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:00.056957 5099 raft_consensus.cc:348] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.057085 5099 raft_consensus.cc:374] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:00.057125 5099 raft_consensus.cc:729] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be87462c243747a69094cf205190dc0d, State: Initialized, Role: FOLLOWER
I20260810 07:20:00.057261 5099 consensus_queue.cc:260] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.057534 5101 tablet_bootstrap.cc:654] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:00.057873 5100 tablet_bootstrap.cc:492] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0: No bootstrap required, opened a new log
I20260810 07:20:00.057962 5100 ts_tablet_manager.cc:1397] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 07:20:00.058051 5099 ts_tablet_manager.cc:1428] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:00.058419 5100 raft_consensus.cc:348] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.058550 5100 raft_consensus.cc:374] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:00.058601 5100 raft_consensus.cc:729] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3353bfe8017145ae83ebe5bd0c10fcb0, State: Initialized, Role: FOLLOWER
I20260810 07:20:00.058753 5100 consensus_queue.cc:260] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.059157 5100 ts_tablet_manager.cc:1428] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260810 07:20:00.059291 5101 tablet_bootstrap.cc:492] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd: No bootstrap required, opened a new log
I20260810 07:20:00.059363 5101 ts_tablet_manager.cc:1397] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 07:20:00.059795 5101 raft_consensus.cc:348] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.059914 5101 raft_consensus.cc:374] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:00.059952 5101 raft_consensus.cc:729] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2909decbff94f0fbe9edcb0980353dd, State: Initialized, Role: FOLLOWER
I20260810 07:20:00.060088 5101 consensus_queue.cc:260] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.060532 5101 ts_tablet_manager.cc:1428] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260810 07:20:00.283217 5051 raft_consensus.cc:482] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:00.283397 5051 raft_consensus.cc:504] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.283929 5051 leader_election.cc:288] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3353bfe8017145ae83ebe5bd0c10fcb0 (127.1.125.194:44947), b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565)
I20260810 07:20:00.284456 4846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d412e5546b1b4a9f89b78f898df41f2b" candidate_uuid: "be87462c243747a69094cf205190dc0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2909decbff94f0fbe9edcb0980353dd" is_pre_election: true
I20260810 07:20:00.284518 4624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d412e5546b1b4a9f89b78f898df41f2b" candidate_uuid: "be87462c243747a69094cf205190dc0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" is_pre_election: true
I20260810 07:20:00.284693 4846 raft_consensus.cc:2454] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be87462c243747a69094cf205190dc0d in term 0.
I20260810 07:20:00.284786 4624 raft_consensus.cc:2454] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be87462c243747a69094cf205190dc0d in term 0.
I20260810 07:20:00.285064 4308 leader_election.cc:302] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2909decbff94f0fbe9edcb0980353dd, be87462c243747a69094cf205190dc0d; no voters:
I20260810 07:20:00.285341 5051 raft_consensus.cc:2781] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:00.285401 5051 raft_consensus.cc:482] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:00.285429 5051 raft_consensus.cc:3037] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:00.287187 5051 raft_consensus.cc:504] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.287546 5051 leader_election.cc:288] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [CANDIDATE]: Term 1 election: Requested vote from peers 3353bfe8017145ae83ebe5bd0c10fcb0 (127.1.125.194:44947), b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565)
I20260810 07:20:00.287966 4624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d412e5546b1b4a9f89b78f898df41f2b" candidate_uuid: "be87462c243747a69094cf205190dc0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0"
I20260810 07:20:00.288038 4846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d412e5546b1b4a9f89b78f898df41f2b" candidate_uuid: "be87462c243747a69094cf205190dc0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2909decbff94f0fbe9edcb0980353dd"
I20260810 07:20:00.288102 4624 raft_consensus.cc:3037] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:00.288169 4846 raft_consensus.cc:3037] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:00.289214 4624 raft_consensus.cc:2454] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be87462c243747a69094cf205190dc0d in term 1.
I20260810 07:20:00.289433 4846 raft_consensus.cc:2454] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be87462c243747a69094cf205190dc0d in term 1.
I20260810 07:20:00.289556 4315 leader_election.cc:302] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3353bfe8017145ae83ebe5bd0c10fcb0, be87462c243747a69094cf205190dc0d; no voters:
I20260810 07:20:00.289849 5051 raft_consensus.cc:2781] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:00.289965 5051 raft_consensus.cc:686] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [term 1 LEADER]: Becoming Leader. State: Replica: be87462c243747a69094cf205190dc0d, State: Running, Role: LEADER
I20260810 07:20:00.290125 5051 consensus_queue.cc:237] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.291939 4168 catalog_manager.cc:5495] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d reported cstate change: term changed from 0 to 1, leader changed from <none> to be87462c243747a69094cf205190dc0d (127.1.125.193). New cstate: current_term: 1 leader_uuid: "be87462c243747a69094cf205190dc0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } health_report { overall_health: UNKNOWN } } }
I20260810 07:20:00.362780 5112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.358806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57856 (local address 127.1.125.193:32885)
0810 07:20:00.359061 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.359068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.359081 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:20:00.359172 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.359174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.359215 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.359284 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:00.359631 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.360145 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.361596 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.361869 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.361953 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.362312 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:00.362385 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:00.362470 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.362570 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.362620 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":113,"threads_started":1}
I20260810 07:20:00.369474 4846 raft_consensus.cc:1261] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Refusing update from remote peer be87462c243747a69094cf205190dc0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:00.369472 4624 raft_consensus.cc:1261] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Refusing update from remote peer be87462c243747a69094cf205190dc0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:00.369848 5057 consensus_queue.cc:1035] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:00.370029 5076 consensus_queue.cc:1035] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:00.404050 4168 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:00.404237 4168 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:00.406270 4168 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20: Sending DeleteTablet for 3 replicas of tablet d412e5546b1b4a9f89b78f898df41f2b
I20260810 07:20:00.406960 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet d412e5546b1b4a9f89b78f898df41f2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:00 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:41560
I20260810 07:20:00.406960 4589 tablet_service.cc:1514] Processing DeleteTablet for tablet d412e5546b1b4a9f89b78f898df41f2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:00 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:44390
I20260810 07:20:00.406970 4384 tablet_service.cc:1514] Processing DeleteTablet for tablet d412e5546b1b4a9f89b78f898df41f2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:00 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:36402
I20260810 07:20:00.407155 5085 tablet_replica.cc:330] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d: stopping tablet replica
I20260810 07:20:00.407167 5087 tablet_replica.cc:330] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0: stopping tablet replica
I20260810 07:20:00.407287 5087 raft_consensus.cc:2229] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:00.407289 5085 raft_consensus.cc:2229] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:00.407419 5087 raft_consensus.cc:2258] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:00.407460 5088 tablet_replica.cc:330] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd: stopping tablet replica
I20260810 07:20:00.407519 5085 raft_consensus.cc:2258] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:00.407598 5088 raft_consensus.cc:2229] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:00.407716 5088 raft_consensus.cc:2258] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:00.408196 5087 ts_tablet_manager.cc:1905] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:00.408531 5088 ts_tablet_manager.cc:1905] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:00.408625 5085 ts_tablet_manager.cc:1905] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:00.410647 5085 ts_tablet_manager.cc:1918] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:20:00.410753 5085 log.cc:1196] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-0/wal/wals/d412e5546b1b4a9f89b78f898df41f2b
I20260810 07:20:00.411015 5085 ts_tablet_manager.cc:1939] T d412e5546b1b4a9f89b78f898df41f2b P be87462c243747a69094cf205190dc0d: Deleting consensus metadata
I20260810 07:20:00.411010 5087 ts_tablet_manager.cc:1918] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:20:00.411000 5088 ts_tablet_manager.cc:1918] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:20:00.411092 5088 log.cc:1196] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/d412e5546b1b4a9f89b78f898df41f2b
I20260810 07:20:00.411092 5087 log.cc:1196] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/d412e5546b1b4a9f89b78f898df41f2b
I20260810 07:20:00.411368 5087 ts_tablet_manager.cc:1939] T d412e5546b1b4a9f89b78f898df41f2b P 3353bfe8017145ae83ebe5bd0c10fcb0: Deleting consensus metadata
I20260810 07:20:00.411368 5088 ts_tablet_manager.cc:1939] T d412e5546b1b4a9f89b78f898df41f2b P b2909decbff94f0fbe9edcb0980353dd: Deleting consensus metadata
I20260810 07:20:00.411738 4150 catalog_manager.cc:4841] TS be87462c243747a69094cf205190dc0d (127.1.125.193:32885): tablet d412e5546b1b4a9f89b78f898df41f2b (table test-table [id=8bd73ac2bbc94270b4ea84f0155a948f]) successfully deleted
I20260810 07:20:00.411808 4168 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:20:00.411991 4150 catalog_manager.cc:4841] TS 3353bfe8017145ae83ebe5bd0c10fcb0 (127.1.125.194:44947): tablet d412e5546b1b4a9f89b78f898df41f2b (table test-table [id=8bd73ac2bbc94270b4ea84f0155a948f]) successfully deleted
I20260810 07:20:00.412114 4146 catalog_manager.cc:4841] TS b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565): tablet d412e5546b1b4a9f89b78f898df41f2b (table test-table [id=8bd73ac2bbc94270b4ea84f0155a948f]) successfully deleted
W20260810 07:20:00.412249 4168 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:00.416702 4812 tablet_service.cc:1467] Processing CreateTablet for tablet 8486c07b815f41b98ca30aca83b9f28d (DEFAULT_TABLE table=test-table [id=b8406062815a447aa69dc4bc122ba307]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:00.416702 4589 tablet_service.cc:1467] Processing CreateTablet for tablet 8486c07b815f41b98ca30aca83b9f28d (DEFAULT_TABLE table=test-table [id=b8406062815a447aa69dc4bc122ba307]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:00.416716 4384 tablet_service.cc:1467] Processing CreateTablet for tablet 8486c07b815f41b98ca30aca83b9f28d (DEFAULT_TABLE table=test-table [id=b8406062815a447aa69dc4bc122ba307]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:00.416967 4589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8486c07b815f41b98ca30aca83b9f28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:00.416967 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8486c07b815f41b98ca30aca83b9f28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:00.416985 4384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8486c07b815f41b98ca30aca83b9f28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:00.418928 5099 tablet_bootstrap.cc:492] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d: Bootstrap starting.
I20260810 07:20:00.419173 5100 tablet_bootstrap.cc:492] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0: Bootstrap starting.
I20260810 07:20:00.419806 5099 tablet_bootstrap.cc:654] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:00.419978 5100 tablet_bootstrap.cc:654] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:00.420549 5101 tablet_bootstrap.cc:492] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd: Bootstrap starting.
I20260810 07:20:00.421113 5099 tablet_bootstrap.cc:492] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d: No bootstrap required, opened a new log
I20260810 07:20:00.421200 5099 ts_tablet_manager.cc:1397] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:20:00.421523 5101 tablet_bootstrap.cc:654] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:00.421628 5099 raft_consensus.cc:348] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.421762 5099 raft_consensus.cc:374] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:00.421806 5099 raft_consensus.cc:729] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be87462c243747a69094cf205190dc0d, State: Initialized, Role: FOLLOWER
I20260810 07:20:00.421943 5099 consensus_queue.cc:260] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.422255 5099 ts_tablet_manager.cc:1428] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:00.422541 5100 tablet_bootstrap.cc:492] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0: No bootstrap required, opened a new log
I20260810 07:20:00.422643 5100 ts_tablet_manager.cc:1397] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260810 07:20:00.423007 5100 raft_consensus.cc:348] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.423122 5100 raft_consensus.cc:374] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:00.423167 5100 raft_consensus.cc:729] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3353bfe8017145ae83ebe5bd0c10fcb0, State: Initialized, Role: FOLLOWER
I20260810 07:20:00.423295 5100 consensus_queue.cc:260] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.423564 5100 ts_tablet_manager.cc:1428] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:20:00.424434 5101 tablet_bootstrap.cc:492] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd: No bootstrap required, opened a new log
I20260810 07:20:00.425230 5101 ts_tablet_manager.cc:1397] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260810 07:20:00.425706 5101 raft_consensus.cc:348] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.425835 5101 raft_consensus.cc:374] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:00.425884 5101 raft_consensus.cc:729] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2909decbff94f0fbe9edcb0980353dd, State: Initialized, Role: FOLLOWER
I20260810 07:20:00.426013 5101 consensus_queue.cc:260] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.426327 5101 ts_tablet_manager.cc:1428] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:20:00.553876 5107 raft_consensus.cc:482] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:00.554049 5107 raft_consensus.cc:504] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.554489 5107 leader_election.cc:288] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be87462c243747a69094cf205190dc0d (127.1.125.193:32885), b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565)
I20260810 07:20:00.555011 4414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8486c07b815f41b98ca30aca83b9f28d" candidate_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be87462c243747a69094cf205190dc0d" is_pre_election: true
I20260810 07:20:00.555011 4846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8486c07b815f41b98ca30aca83b9f28d" candidate_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2909decbff94f0fbe9edcb0980353dd" is_pre_election: true
I20260810 07:20:00.555195 4846 raft_consensus.cc:2454] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3353bfe8017145ae83ebe5bd0c10fcb0 in term 0.
I20260810 07:20:00.555195 4414 raft_consensus.cc:2454] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3353bfe8017145ae83ebe5bd0c10fcb0 in term 0.
I20260810 07:20:00.555560 4531 leader_election.cc:302] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3353bfe8017145ae83ebe5bd0c10fcb0, b2909decbff94f0fbe9edcb0980353dd, be87462c243747a69094cf205190dc0d; no voters:
I20260810 07:20:00.555799 5107 raft_consensus.cc:2781] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:00.555895 5107 raft_consensus.cc:482] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:00.555951 5107 raft_consensus.cc:3037] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:00.557216 5107 raft_consensus.cc:504] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.557559 5107 leader_election.cc:288] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [CANDIDATE]: Term 1 election: Requested vote from peers be87462c243747a69094cf205190dc0d (127.1.125.193:32885), b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565)
I20260810 07:20:00.557940 4846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8486c07b815f41b98ca30aca83b9f28d" candidate_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2909decbff94f0fbe9edcb0980353dd"
I20260810 07:20:00.558058 4846 raft_consensus.cc:3037] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:00.558013 4414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8486c07b815f41b98ca30aca83b9f28d" candidate_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be87462c243747a69094cf205190dc0d"
I20260810 07:20:00.558195 4414 raft_consensus.cc:3037] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:00.559104 4846 raft_consensus.cc:2454] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3353bfe8017145ae83ebe5bd0c10fcb0 in term 1.
I20260810 07:20:00.559181 4414 raft_consensus.cc:2454] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3353bfe8017145ae83ebe5bd0c10fcb0 in term 1.
I20260810 07:20:00.559432 4527 leader_election.cc:302] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3353bfe8017145ae83ebe5bd0c10fcb0, b2909decbff94f0fbe9edcb0980353dd; no voters:
I20260810 07:20:00.559628 5107 raft_consensus.cc:2781] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:00.559823 5107 raft_consensus.cc:686] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 LEADER]: Becoming Leader. State: Replica: 3353bfe8017145ae83ebe5bd0c10fcb0, State: Running, Role: LEADER
I20260810 07:20:00.560000 5107 consensus_queue.cc:237] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } }
I20260810 07:20:00.561849 4168 catalog_manager.cc:5495] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3353bfe8017145ae83ebe5bd0c10fcb0 (127.1.125.194). New cstate: current_term: 1 leader_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3353bfe8017145ae83ebe5bd0c10fcb0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 } health_report { overall_health: UNKNOWN } } }
I20260810 07:20:00.582609 5124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:00.579276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38022 (local address 127.1.125.194:44947)
0810 07:20:00.579605 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:00.579614 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:00.579626 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:00.579669 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:00.579671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:00.579711 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:00.579804 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:00.580082 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.580685 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.581540 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:00.581755 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:00.581861 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:00.582138 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:00.582212 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:00.582295 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:00.582389 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 07:20:00.582441 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":107,"threads_started":1}
I20260810 07:20:00.588649 4846 raft_consensus.cc:1261] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Refusing update from remote peer 3353bfe8017145ae83ebe5bd0c10fcb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:00.588649 4414 raft_consensus.cc:1261] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d [term 1 FOLLOWER]: Refusing update from remote peer 3353bfe8017145ae83ebe5bd0c10fcb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:00.589071 5122 consensus_queue.cc:1035] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2909decbff94f0fbe9edcb0980353dd" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:00.589224 5107 consensus_queue.cc:1035] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be87462c243747a69094cf205190dc0d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:20:00.602859 5060 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 07:20:02.156531 4167 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:02.156693 4167 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:02.159054 4167 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a156e283e774438e9644e555d31c2a20: Sending DeleteTablet for 3 replicas of tablet 8486c07b815f41b98ca30aca83b9f28d
I20260810 07:20:02.159538 4384 tablet_service.cc:1514] Processing DeleteTablet for tablet 8486c07b815f41b98ca30aca83b9f28d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:02 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:36402
I20260810 07:20:02.159616 4589 tablet_service.cc:1514] Processing DeleteTablet for tablet 8486c07b815f41b98ca30aca83b9f28d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:02 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:44390
I20260810 07:20:02.159657 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 8486c07b815f41b98ca30aca83b9f28d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:02 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:41560
I20260810 07:20:02.159952 5165 tablet_replica.cc:330] T 8486c07b815f41b98ca30aca83b9f28d P be87462c243747a69094cf205190dc0d: stopping tablet replica
I20260810 07:20:02.159967 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 4278
I20260810 07:20:02.173453 5167 tablet_replica.cc:330] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0: stopping tablet replica
I20260810 07:20:02.173676 5167 raft_consensus.cc:2229] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:02.173954 5167 raft_consensus.cc:2258] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:02.174288 5170 tablet_replica.cc:330] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd: stopping tablet replica
I20260810 07:20:02.174455 5170 raft_consensus.cc:2229] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:02.174603 5170 raft_consensus.cc:2258] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:02.174839 5167 ts_tablet_manager.cc:1905] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:02.175491 5170 ts_tablet_manager.cc:1905] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:02.177983 5167 ts_tablet_manager.cc:1918] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:20:02.178078 5167 log.cc:1196] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/8486c07b815f41b98ca30aca83b9f28d
I20260810 07:20:02.178390 5167 ts_tablet_manager.cc:1939] T 8486c07b815f41b98ca30aca83b9f28d P 3353bfe8017145ae83ebe5bd0c10fcb0: Deleting consensus metadata
I20260810 07:20:02.178390 5170 ts_tablet_manager.cc:1918] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:20:02.178474 5170 log.cc:1196] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/8486c07b815f41b98ca30aca83b9f28d
I20260810 07:20:02.178822 5170 ts_tablet_manager.cc:1939] T 8486c07b815f41b98ca30aca83b9f28d P b2909decbff94f0fbe9edcb0980353dd: Deleting consensus metadata
I20260810 07:20:02.179335 4150 catalog_manager.cc:4841] TS 3353bfe8017145ae83ebe5bd0c10fcb0 (127.1.125.194:44947): tablet 8486c07b815f41b98ca30aca83b9f28d (table test-table [id=b8406062815a447aa69dc4bc122ba307]) successfully deleted
I20260810 07:20:02.179576 4146 catalog_manager.cc:4841] TS b2909decbff94f0fbe9edcb0980353dd (127.1.125.195:40565): tablet 8486c07b815f41b98ca30aca83b9f28d (table test-table [id=b8406062815a447aa69dc4bc122ba307]) successfully deleted
W20260810 07:20:02.183122 4150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:32885 (error 108)
I20260810 07:20:02.183525 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 4504
I20260810 07:20:02.186951 5173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:02.186357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:32885 (local address 127.0.0.1:57872)
0810 07:20:02.186671 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:02.186800 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:32885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":98,"threads_started":1}
W20260810 07:20:02.187229 4150 catalog_manager.cc:4584] TS be87462c243747a69094cf205190dc0d (127.1.125.193:32885): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8486c07b815f41b98ca30aca83b9f28d: Network error: Client connection negotiation failed: client connection to 127.1.125.193:32885: connect: Connection refused (error 111)
I20260810 07:20:02.195513 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 4712
I20260810 07:20:02.209947 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 4122
2026-08-10T07:20:02Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6972 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:02 dist-test-slave-44d2 krb5kdc[5177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:02 dist-test-slave-44d2 krb5kdc[5177](info): set up 2 sockets
Aug 10 07:20:02 dist-test-slave-44d2 krb5kdc[5177](info): commencing operation
krb5kdc: starting...
W20260810 07:20:03.497439 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:20:03 dist-test-slave-44d2 krb5kdc[5177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:20:03.627991 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:45965
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:33633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:45965
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:03.741426 5195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:03.741703 5195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:03.741776 5195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:03.745731 5195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:03.745811 5195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:03.745850 5195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:03.745908 5195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:03.745949 5195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:03.751081 5195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:33633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:45965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:45965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:03.752125 5195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:03.753199 5195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:03.762015 5205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:03.762152 5195 server_base.cc:1034] running on GCE node
W20260810 07:20:03.762086 5204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:03.762048 5208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:03.762856 5195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:03.763875 5195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:03.765064 5195 hybrid_clock.cc:648] HybridClock initialized: now 1786346403765043 us; error 45 us; skew 500 ppm
Aug 10 07:20:03 dist-test-slave-44d2 krb5kdc[5177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346403, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:03.767762 5195 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:20:03.768738 5195 webserver.cc:460] Webserver started at http://127.1.125.254:43149/ using document root <none> and password file <none>
I20260810 07:20:03.769300 5195 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:03.769392 5195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:03.769640 5195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:03.771667 5195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "de78cb366eb94b9eabcf5dfb883dc734"
format_stamp: "Formatted at 2026-08-10 07:20:03 on dist-test-slave-44d2"
server_key: "bf7f84107dac866a5bae8883ad516dcb"
server_key_iv: "fdc6e3ca6814b46963fc467973135a03"
server_key_version: "encryptionkey@0"
I20260810 07:20:03.772228 5195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "de78cb366eb94b9eabcf5dfb883dc734"
format_stamp: "Formatted at 2026-08-10 07:20:03 on dist-test-slave-44d2"
server_key: "bf7f84107dac866a5bae8883ad516dcb"
server_key_iv: "fdc6e3ca6814b46963fc467973135a03"
server_key_version: "encryptionkey@0"
I20260810 07:20:03.775951 5195 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:20:03.778354 5221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:03.779578 5195 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:03.779734 5195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "de78cb366eb94b9eabcf5dfb883dc734"
format_stamp: "Formatted at 2026-08-10 07:20:03 on dist-test-slave-44d2"
server_key: "bf7f84107dac866a5bae8883ad516dcb"
server_key_iv: "fdc6e3ca6814b46963fc467973135a03"
server_key_version: "encryptionkey@0"
I20260810 07:20:03.779845 5195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:03.800220 5195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:03.803133 5195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:03.803349 5195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:03.810734 5195 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:45965
I20260810 07:20:03.810763 5299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:45965 every 8 connection(s)
I20260810 07:20:03.811676 5195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:03.814139 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 5195
I20260810 07:20:03.814241 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:03.814532 1527 external_mini_cluster.cc:1428] Setting key 9555ae3a5786ac407184a2a9877b47e1
I20260810 07:20:03.814605 5303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:03.822238 5303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734: Bootstrap starting.
Aug 10 07:20:03 dist-test-slave-44d2 krb5kdc[5177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:03.824887 5303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:03.825767 5303 log.cc:824] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:03.828292 5303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734: No bootstrap required, opened a new log
I20260810 07:20:03.831056 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:03.815993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.1.125.254:45965)
0810 07:20:03.816435 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:03.816456 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:03.816537 (+ 81us) server_negotiation.cc:407] Connection header received
0810 07:20:03.817551 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:03.817586 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:03.818017 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:03.818400 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:03.819077 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:03.820812 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:03.821666 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:03.822355 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:03.822514 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:03.825378 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:03.825383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:03.825396 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:03.825424 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:03.827611 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:03.828054 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:03.828061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:03.828067 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:03.828217 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:03.828612 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:03.828615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:03.828617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:03.829157 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:03.829351 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:03.829612 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 07:20:03.829917 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":121,"threads_started":1}
I20260810 07:20:03.833053 5303 raft_consensus.cc:348] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de78cb366eb94b9eabcf5dfb883dc734" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45965 } }
I20260810 07:20:03.833338 5303 raft_consensus.cc:374] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:03.833446 5303 raft_consensus.cc:729] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de78cb366eb94b9eabcf5dfb883dc734, State: Initialized, Role: FOLLOWER
I20260810 07:20:03.833881 5303 consensus_queue.cc:260] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de78cb366eb94b9eabcf5dfb883dc734" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45965 } }
I20260810 07:20:03.834041 5303 raft_consensus.cc:388] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:03.834129 5303 raft_consensus.cc:482] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:03.834236 5303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:03.835187 5303 raft_consensus.cc:504] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de78cb366eb94b9eabcf5dfb883dc734" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45965 } }
I20260810 07:20:03.835538 5303 leader_election.cc:302] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de78cb366eb94b9eabcf5dfb883dc734; no voters:
I20260810 07:20:03.835858 5303 leader_election.cc:288] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:03.836030 5311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:03.836277 5311 raft_consensus.cc:686] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [term 1 LEADER]: Becoming Leader. State: Replica: de78cb366eb94b9eabcf5dfb883dc734, State: Running, Role: LEADER
I20260810 07:20:03.836658 5311 consensus_queue.cc:237] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de78cb366eb94b9eabcf5dfb883dc734" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45965 } }
I20260810 07:20:03.837153 5303 sys_catalog.cc:564] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:03.838363 5312 sys_catalog.cc:455] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de78cb366eb94b9eabcf5dfb883dc734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de78cb366eb94b9eabcf5dfb883dc734" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45965 } } }
I20260810 07:20:03.838521 5312 sys_catalog.cc:458] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:03.838884 5313 sys_catalog.cc:455] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de78cb366eb94b9eabcf5dfb883dc734. Latest consensus state: current_term: 1 leader_uuid: "de78cb366eb94b9eabcf5dfb883dc734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de78cb366eb94b9eabcf5dfb883dc734" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45965 } } }
I20260810 07:20:03.839514 5313 sys_catalog.cc:458] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:03.839903 5321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:03.842655 5321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:03.848117 5321 catalog_manager.cc:1347] Generated new cluster ID: 7c64c61960954f059965db0f3da5ebf6
I20260810 07:20:03.848194 5321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:03.868856 5321 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:03.869801 5321 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:03.875200 5321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de78cb366eb94b9eabcf5dfb883dc734: Generated new TSK 0
I20260810 07:20:03.875988 5321 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:20:03.907925 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45965
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:33633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:04.023473 5336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:04.023757 5336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:04.023833 5336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:04.027737 5336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:04.027822 5336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:04.027911 5336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:04.032938 5336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:33633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:04.033936 5336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:04.034947 5336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:04.044612 5348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:04.044612 5343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:04.044633 5344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:04.045073 5336 server_base.cc:1034] running on GCE node
I20260810 07:20:04.045483 5336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:04.046149 5336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:04.047309 5336 hybrid_clock.cc:648] HybridClock initialized: now 1786346404047294 us; error 26 us; skew 500 ppm
Aug 10 07:20:04 dist-test-slave-44d2 krb5kdc[5177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:04.050221 5336 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:20:04.051064 5336 webserver.cc:460] Webserver started at http://127.1.125.193:44679/ using document root <none> and password file <none>
I20260810 07:20:04.051626 5336 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:04.051687 5336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:04.051977 5336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:04.054538 5336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "14775686c3814cdba5c7c103aea8c09f"
format_stamp: "Formatted at 2026-08-10 07:20:04 on dist-test-slave-44d2"
server_key: "a75a9e3bdcaff91038d42e65d7140bc9"
server_key_iv: "db9aa3b8950fb63bff5383e131d5518a"
server_key_version: "encryptionkey@0"
I20260810 07:20:04.055117 5336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "14775686c3814cdba5c7c103aea8c09f"
format_stamp: "Formatted at 2026-08-10 07:20:04 on dist-test-slave-44d2"
server_key: "a75a9e3bdcaff91038d42e65d7140bc9"
server_key_iv: "db9aa3b8950fb63bff5383e131d5518a"
server_key_version: "encryptionkey@0"
I20260810 07:20:04.058817 5336 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 07:20:04.061177 5357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:04.062306 5336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:20:04.062441 5336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "14775686c3814cdba5c7c103aea8c09f"
format_stamp: "Formatted at 2026-08-10 07:20:04 on dist-test-slave-44d2"
server_key: "a75a9e3bdcaff91038d42e65d7140bc9"
server_key_iv: "db9aa3b8950fb63bff5383e131d5518a"
server_key_version: "encryptionkey@0"
I20260810 07:20:04.062558 5336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:04.074231 5336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:04.077556 5336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:04.077762 5336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:04.078341 5336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:04.079161 5336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:04.079236 5336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:04.079308 5336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:04.079353 5336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:04.089103 5336 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:44761
I20260810 07:20:04.089193 5537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:44761 every 8 connection(s)
I20260810 07:20:04.090005 5336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:04.094856 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 5336
I20260810 07:20:04.094947 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:04.095250 1527 external_mini_cluster.cc:1428] Setting key 8d70b411f685d33a12fe044ffd3e21e3
Aug 10 07:20:04 dist-test-slave-44d2 krb5kdc[5177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:04.105720 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:04.091862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:57577 (local address 127.1.125.254:45965)
0810 07:20:04.092661 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:04.092668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:04.093037 (+ 369us) server_negotiation.cc:407] Connection header received
0810 07:20:04.094429 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:04.094433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:04.094501 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:04.094611 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:04.096004 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:04.096656 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:04.098425 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:04.098661 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:04.098769 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:04.101743 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:04.101746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:04.101748 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:04.101772 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:04.103322 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:04.104008 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:04.104010 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:04.104012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:04.104090 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:04.104563 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:04.104566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:04.104567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:04.104740 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:04.104818 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:04.105491 (+ 673us) server_negotiation.cc:299] Negotiation successful
0810 07:20:04.105596 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687}
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:20:04.106978 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:04.092324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45965 (local address 127.1.125.193:57577)
0810 07:20:04.092882 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:04.092924 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:04.094211 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:04.094793 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:04.094800 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:04.095234 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:04.095860 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:04.095877 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:04.096821 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:04.096825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:04.098279 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:04.098296 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:04.098822 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:04.098830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:04.099070 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:04.099695 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:04.099712 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:04.101549 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:04.103453 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:04.103462 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:04.103480 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:04.103901 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:04.104198 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:04.104205 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:04.104208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:04.104404 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:04.104855 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:04.104864 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:04.105205 (+ 341us) client_negotiation.cc:769] Sending connection context
0810 07:20:04.105436 (+ 231us) client_negotiation.cc:240] Negotiation successful
0810 07:20:04.105814 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":186,"threads_started":1}
I20260810 07:20:04.108258 5538 heartbeater.cc:347] Connected to a master server at 127.1.125.254:45965
I20260810 07:20:04.108531 5538 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:04.108991 5538 heartbeater.cc:511] Master 127.1.125.254:45965 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
I20260810 07:20:04.110617 5247 ts_manager.cc:194] Registered new tserver with Master: 14775686c3814cdba5c7c103aea8c09f (127.1.125.193:44761)
I20260810 07:20:04.111775 5247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:57577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:20:04.127451 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45965
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:33633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:04.242821 5551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:04.243129 5551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:04.243227 5551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:04.247370 5551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:04.247468 5551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:04.247558 5551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:04.253047 5551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:33633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:04.254202 5551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:04.255288 5551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:04.265919 5558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:04.265923 5566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:04.266000 5559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:04.266166 5551 server_base.cc:1034] running on GCE node
I20260810 07:20:04.266693 5551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:04.267406 5551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:04.268570 5551 hybrid_clock.cc:648] HybridClock initialized: now 1786346404268553 us; error 30 us; skew 500 ppm
Aug 10 07:20:04 dist-test-slave-44d2 krb5kdc[5177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:04.271586 5551 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:20:04.272531 5551 webserver.cc:460] Webserver started at http://127.1.125.194:38887/ using document root <none> and password file <none>
I20260810 07:20:04.273149 5551 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:04.273243 5551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:04.273514 5551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:04.275600 5551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "a459af0a71ed4a559b50c02369de4046"
format_stamp: "Formatted at 2026-08-10 07:20:04 on dist-test-slave-44d2"
server_key: "3b4b94f70f0ce05d5ddad2872aade00f"
server_key_iv: "4e63dd49b8d15ecbea52e390db565f79"
server_key_version: "encryptionkey@0"
I20260810 07:20:04.276185 5551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "a459af0a71ed4a559b50c02369de4046"
format_stamp: "Formatted at 2026-08-10 07:20:04 on dist-test-slave-44d2"
server_key: "3b4b94f70f0ce05d5ddad2872aade00f"
server_key_iv: "4e63dd49b8d15ecbea52e390db565f79"
server_key_version: "encryptionkey@0"
I20260810 07:20:04.280066 5551 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 07:20:04.282572 5576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:04.283833 5551 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:20:04.283978 5551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "a459af0a71ed4a559b50c02369de4046"
format_stamp: "Formatted at 2026-08-10 07:20:04 on dist-test-slave-44d2"
server_key: "3b4b94f70f0ce05d5ddad2872aade00f"
server_key_iv: "4e63dd49b8d15ecbea52e390db565f79"
server_key_version: "encryptionkey@0"
I20260810 07:20:04.284091 5551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:04.299499 5551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:04.303045 5551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:04.303263 5551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:04.303901 5551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:04.304782 5551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:04.304855 5551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:04.304929 5551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:04.304972 5551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:04.314235 5551 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:37655
I20260810 07:20:04.314318 5767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:37655 every 8 connection(s)
I20260810 07:20:04.315183 5551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:04.324431 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 5551
I20260810 07:20:04.324600 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:04.324942 1527 external_mini_cluster.cc:1428] Setting key 1161bedd2526ca7777f0f8ad0087ca25
Aug 10 07:20:04 dist-test-slave-44d2 krb5kdc[5177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:04.330713 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:04.317288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:56701 (local address 127.1.125.254:45965)
0810 07:20:04.317454 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:04.317459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:04.318389 (+ 930us) server_negotiation.cc:407] Connection header received
0810 07:20:04.319712 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:04.319715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:04.319763 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:04.319840 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:04.321221 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:04.321763 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:04.322877 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:04.323224 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:04.323352 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:04.326395 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:04.326398 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:04.326400 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:04.326422 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:04.328187 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:04.328874 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:04.328877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:04.328878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:04.328958 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:04.329331 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:04.329334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:04.329336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:04.329526 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:04.329637 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:04.330303 (+ 666us) server_negotiation.cc:299] Negotiation successful
0810 07:20:04.330464 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 07:20:04.332389 5772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:04.317735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45965 (local address 127.1.125.194:56701)
0810 07:20:04.318223 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:04.318269 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:04.319502 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:04.320017 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:04.320024 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:04.320424 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:04.321068 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:04.321093 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:04.321875 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:04.321877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:04.322768 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:04.322778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:04.323430 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:04.323436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:04.323715 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:04.324350 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:04.324374 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:04.326198 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:04.328329 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:04.328333 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:04.328363 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:04.328745 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:04.329072 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:04.329077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:04.329079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:04.329222 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:04.329636 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:04.329643 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:04.329974 (+ 331us) client_negotiation.cc:769] Sending connection context
0810 07:20:04.330883 (+ 909us) client_negotiation.cc:240] Negotiation successful
0810 07:20:04.331258 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":122,"threads_started":1}
I20260810 07:20:04.333767 5768 heartbeater.cc:347] Connected to a master server at 127.1.125.254:45965
I20260810 07:20:04.334038 5768 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:04.334563 5768 heartbeater.cc:511] Master 127.1.125.254:45965 requested a full tablet report, sending...
I20260810 07:20:04.335711 5247 ts_manager.cc:194] Registered new tserver with Master: a459af0a71ed4a559b50c02369de4046 (127.1.125.194:37655)
I20260810 07:20:04.336288 5247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:56701
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:20:04.355602 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45965
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:33633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:04.472092 5776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:04.472352 5776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:04.472395 5776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:04.476409 5776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:04.476473 5776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:04.476608 5776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:04.481945 5776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:33633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:04.482980 5776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:04.484380 5776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:04.494433 5788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:04.494441 5789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:04.494655 5776 server_base.cc:1034] running on GCE node
W20260810 07:20:04.494804 5794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:04.495261 5776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:04.496155 5776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:04.497341 5776 hybrid_clock.cc:648] HybridClock initialized: now 1786346404497297 us; error 60 us; skew 500 ppm
Aug 10 07:20:04 dist-test-slave-44d2 krb5kdc[5177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:04.500073 5776 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:20:04.501086 5776 webserver.cc:460] Webserver started at http://127.1.125.195:40913/ using document root <none> and password file <none>
I20260810 07:20:04.501672 5776 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:04.501770 5776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:04.501991 5776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:04.504103 5776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "5401399a09444ef9bd03b802172a73cc"
format_stamp: "Formatted at 2026-08-10 07:20:04 on dist-test-slave-44d2"
server_key: "a161bf23db9d76e043e957bfbe3638e9"
server_key_iv: "778242d8076f56c54bdbf5c491c8dfa5"
server_key_version: "encryptionkey@0"
I20260810 07:20:04.504740 5776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "5401399a09444ef9bd03b802172a73cc"
format_stamp: "Formatted at 2026-08-10 07:20:04 on dist-test-slave-44d2"
server_key: "a161bf23db9d76e043e957bfbe3638e9"
server_key_iv: "778242d8076f56c54bdbf5c491c8dfa5"
server_key_version: "encryptionkey@0"
I20260810 07:20:04.508764 5776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 07:20:04.511837 5805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:04.513121 5776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:04.513268 5776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "5401399a09444ef9bd03b802172a73cc"
format_stamp: "Formatted at 2026-08-10 07:20:04 on dist-test-slave-44d2"
server_key: "a161bf23db9d76e043e957bfbe3638e9"
server_key_iv: "778242d8076f56c54bdbf5c491c8dfa5"
server_key_version: "encryptionkey@0"
I20260810 07:20:04.513391 5776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:04.566120 5776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:04.569252 5776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:04.569473 5776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:04.570127 5776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:04.571009 5776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:04.571094 5776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:04.571173 5776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:04.571233 5776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:04.581007 5776 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:34471
I20260810 07:20:04.581051 5986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:34471 every 8 connection(s)
I20260810 07:20:04.581935 5776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:04.583472 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 5776
I20260810 07:20:04.583566 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:04.583985 1527 external_mini_cluster.cc:1428] Setting key 8b4b9509f1b75cca69c37d95941c12c3
Aug 10 07:20:04 dist-test-slave-44d2 krb5kdc[5177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:04.595875 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:04.583899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:50223 (local address 127.1.125.254:45965)
0810 07:20:04.584449 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:04.584455 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:04.585111 (+ 656us) server_negotiation.cc:407] Connection header received
0810 07:20:04.586463 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:04.586466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:04.586512 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:04.586588 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:04.587795 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:04.588403 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:04.589567 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:04.589795 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:04.589877 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:04.592469 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:04.592472 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:04.592474 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:04.592553 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:04.594038 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:04.594588 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:04.594590 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:04.594591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:04.594663 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:04.594959 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:04.594961 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:04.594962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:04.595138 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:04.595215 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:04.595658 (+ 443us) server_negotiation.cc:299] Negotiation successful
0810 07:20:04.595755 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420}
I20260810 07:20:04.596935 5990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:04.584386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45965 (local address 127.1.125.195:50223)
0810 07:20:04.584942 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:04.585007 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:04.586276 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:04.586714 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:04.586721 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:04.587074 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:04.587670 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:04.587686 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:04.588539 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:04.588542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:04.589460 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:04.589470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:04.589922 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:04.589925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:04.590079 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:04.590701 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:04.590717 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:04.592318 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:04.594220 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:04.594225 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:04.594237 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:04.594496 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:04.594745 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:04.594747 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:04.594748 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:04.594877 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:04.595228 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:04.595234 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:04.595453 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 07:20:04.595620 (+ 167us) client_negotiation.cc:240] Negotiation successful
0810 07:20:04.595863 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":146,"threads_started":1}
I20260810 07:20:04.598306 5987 heartbeater.cc:347] Connected to a master server at 127.1.125.254:45965
I20260810 07:20:04.598526 5987 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:04.598971 5987 heartbeater.cc:511] Master 127.1.125.254:45965 requested a full tablet report, sending...
I20260810 07:20:04.599972 5244 ts_manager.cc:194] Registered new tserver with Master: 5401399a09444ef9bd03b802172a73cc (127.1.125.195:34471)
I20260810 07:20:04.600636 5244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:50223
I20260810 07:20:04.609826 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 07:20:04.625120 5306 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:04.623441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59578 (local address 127.1.125.254:45965)
0810 07:20:04.623618 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:04.623624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:04.623781 (+ 157us) server_negotiation.cc:407] Connection header received
0810 07:20:04.623958 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:04.623961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:04.624023 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:04.624120 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:04.624947 (+ 827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59578: BlockingRecv error: recv got EOF from 127.0.0.1:59578 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 07:20:04.626106 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 5336
I20260810 07:20:04.649309 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 5551
I20260810 07:20:04.659724 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 5776
I20260810 07:20:04.678524 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 5195
2026-08-10T07:20:04Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2465 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:04 dist-test-slave-44d2 krb5kdc[6005](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:04 dist-test-slave-44d2 krb5kdc[6005](info): set up 2 sockets
Aug 10 07:20:04 dist-test-slave-44d2 krb5kdc[6005](info): commencing operation
krb5kdc: starting...
W20260810 07:20:05.987402 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:20:06 dist-test-slave-44d2 krb5kdc[6005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:20:06.118029 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:40205
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:40205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:06.233896 6021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:06.234241 6021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:06.234318 6021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:06.238348 6021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:06.238404 6021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:06.238422 6021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:06.238440 6021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:06.238454 6021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:06.243630 6021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:40205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:40205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:06.244853 6021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:06.245918 6021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:06.254673 6031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:06.254673 6026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:06.254733 6028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:06.255026 6021 server_base.cc:1034] running on GCE node
I20260810 07:20:06.255626 6021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:06.256861 6021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:06.258028 6021 hybrid_clock.cc:648] HybridClock initialized: now 1786346406257992 us; error 46 us; skew 500 ppm
Aug 10 07:20:06 dist-test-slave-44d2 krb5kdc[6005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346406, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:06.261060 6021 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:20:06.261890 6021 webserver.cc:460] Webserver started at http://127.1.125.254:37143/ using document root <none> and password file <none>
I20260810 07:20:06.262457 6021 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:06.262526 6021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:06.262701 6021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:06.264815 6021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "a19e5706c0fd418c9c950e44879593ab"
format_stamp: "Formatted at 2026-08-10 07:20:06 on dist-test-slave-44d2"
server_key: "db3104f2de36dfea900b9214e7942567"
server_key_iv: "c3446223a533edafa09dd6a3dcd12150"
server_key_version: "encryptionkey@0"
I20260810 07:20:06.265336 6021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "a19e5706c0fd418c9c950e44879593ab"
format_stamp: "Formatted at 2026-08-10 07:20:06 on dist-test-slave-44d2"
server_key: "db3104f2de36dfea900b9214e7942567"
server_key_iv: "c3446223a533edafa09dd6a3dcd12150"
server_key_version: "encryptionkey@0"
I20260810 07:20:06.268963 6021 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:20:06.271333 6041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:06.272710 6021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:06.272854 6021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "a19e5706c0fd418c9c950e44879593ab"
format_stamp: "Formatted at 2026-08-10 07:20:06 on dist-test-slave-44d2"
server_key: "db3104f2de36dfea900b9214e7942567"
server_key_iv: "c3446223a533edafa09dd6a3dcd12150"
server_key_version: "encryptionkey@0"
I20260810 07:20:06.272969 6021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:06.311161 6021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:06.314304 6021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:06.314533 6021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:06.321851 6021 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:40205
I20260810 07:20:06.322153 6125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:40205 every 8 connection(s)
I20260810 07:20:06.322921 6021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:06.325601 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 6021
I20260810 07:20:06.325736 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:06.326069 1527 external_mini_cluster.cc:1428] Setting key f11b2ed8f41cf5c0ba21b83ecdbe0f4d
I20260810 07:20:06.326195 6126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:06.333743 6126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab: Bootstrap starting.
Aug 10 07:20:06 dist-test-slave-44d2 krb5kdc[6005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:06.336413 6126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:06.337425 6126 log.cc:824] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:06.339880 6126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab: No bootstrap required, opened a new log
I20260810 07:20:06.341557 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:06.327378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.1.125.254:40205)
0810 07:20:06.327831 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:06.327851 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:06.327893 (+ 42us) server_negotiation.cc:407] Connection header received
0810 07:20:06.328751 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:06.328775 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:06.329208 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:06.329613 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:06.330353 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.331855 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.332899 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.333571 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.333731 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.336443 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:06.336449 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:06.336467 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:06.336515 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:06.338450 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:06.338870 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:06.338876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:06.338882 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:06.339021 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:06.339287 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:06.339290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:06.339291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:06.339795 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:06.340002 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:06.340296 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 07:20:06.340633 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":128,"threads_started":1}
I20260810 07:20:06.344311 6126 raft_consensus.cc:348] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19e5706c0fd418c9c950e44879593ab" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40205 } }
I20260810 07:20:06.344631 6126 raft_consensus.cc:374] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:06.344661 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:20:06.344712 6126 raft_consensus.cc:729] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a19e5706c0fd418c9c950e44879593ab, State: Initialized, Role: FOLLOWER
I20260810 07:20:06.345170 6126 consensus_queue.cc:260] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19e5706c0fd418c9c950e44879593ab" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40205 } }
I20260810 07:20:06.345321 6126 raft_consensus.cc:388] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:06.345402 6126 raft_consensus.cc:482] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:06.345501 6126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:06.346395 6126 raft_consensus.cc:504] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19e5706c0fd418c9c950e44879593ab" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40205 } }
I20260810 07:20:06.346745 6126 leader_election.cc:302] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a19e5706c0fd418c9c950e44879593ab; no voters:
I20260810 07:20:06.347107 6126 leader_election.cc:288] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:06.347179 6138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:06.347456 6138 raft_consensus.cc:686] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [term 1 LEADER]: Becoming Leader. State: Replica: a19e5706c0fd418c9c950e44879593ab, State: Running, Role: LEADER
I20260810 07:20:06.347815 6138 consensus_queue.cc:237] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19e5706c0fd418c9c950e44879593ab" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40205 } }
I20260810 07:20:06.348506 6126 sys_catalog.cc:564] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:06.349066 6143 sys_catalog.cc:455] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a19e5706c0fd418c9c950e44879593ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19e5706c0fd418c9c950e44879593ab" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40205 } } }
I20260810 07:20:06.349272 6143 sys_catalog.cc:458] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:06.349495 6142 sys_catalog.cc:455] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader a19e5706c0fd418c9c950e44879593ab. Latest consensus state: current_term: 1 leader_uuid: "a19e5706c0fd418c9c950e44879593ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19e5706c0fd418c9c950e44879593ab" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40205 } } }
I20260810 07:20:06.349624 6142 sys_catalog.cc:458] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:06.349898 6151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:06.353528 6151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:06.359018 6151 catalog_manager.cc:1347] Generated new cluster ID: 325ac7316ea742abb1968bb54d02a768
I20260810 07:20:06.359091 6151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:06.367355 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:06.361584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48094 (local address 127.1.125.254:40205)
0810 07:20:06.361700 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:06.361705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:06.362041 (+ 336us) server_negotiation.cc:407] Connection header received
0810 07:20:06.362187 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:06.362191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:06.362235 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:06.362322 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:06.362947 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.363741 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.365012 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:06.365359 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:06.365452 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:06.366395 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:06.366399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:06.366402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:06.366906 (+ 504us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:06.366937 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:06.366956 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:06.367039 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:06.367167 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 07:20:06.367222 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:20:06.393891 6151 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:06.394928 6151 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:06.408244 6151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a19e5706c0fd418c9c950e44879593ab: Generated new TSK 0
I20260810 07:20:06.409211 6151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:20:06.422849 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 6021
2026-08-10T07:20:06Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1739 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:06 dist-test-slave-44d2 krb5kdc[6181](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:06 dist-test-slave-44d2 krb5kdc[6181](info): set up 2 sockets
Aug 10 07:20:06 dist-test-slave-44d2 krb5kdc[6181](info): commencing operation
krb5kdc: starting...
W20260810 07:20:07.682889 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:20:07 dist-test-slave-44d2 krb5kdc[6181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:20:07.812016 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:35931
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:35931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:07.926469 6195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:07.926833 6195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:07.926954 6195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:07.931619 6195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:07.931723 6195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:07.931772 6195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:07.931852 6195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:07.931901 6195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:07.937739 6195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:35931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:35931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:07.939006 6195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:07.940237 6195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:07.949800 6204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:07.949872 6208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:07.949807 6210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:07.950309 6195 server_base.cc:1034] running on GCE node
I20260810 07:20:07.950897 6195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:07.952021 6195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:07.953204 6195 hybrid_clock.cc:648] HybridClock initialized: now 1786346407953185 us; error 32 us; skew 500 ppm
Aug 10 07:20:07 dist-test-slave-44d2 krb5kdc[6181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346407, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:07.956458 6195 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:20:07.957532 6195 webserver.cc:460] Webserver started at http://127.1.125.254:41935/ using document root <none> and password file <none>
I20260810 07:20:07.958142 6195 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:07.958206 6195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:07.958444 6195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:07.960608 6195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "3cf424168ff74c499e9d8b4344d174d3"
format_stamp: "Formatted at 2026-08-10 07:20:07 on dist-test-slave-44d2"
server_key: "8cf45f0e17940b2d0e727d22d80edcd7"
server_key_iv: "3a42cae8b4f0294a8416179401d14ad8"
server_key_version: "encryptionkey@0"
I20260810 07:20:07.961201 6195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "3cf424168ff74c499e9d8b4344d174d3"
format_stamp: "Formatted at 2026-08-10 07:20:07 on dist-test-slave-44d2"
server_key: "8cf45f0e17940b2d0e727d22d80edcd7"
server_key_iv: "3a42cae8b4f0294a8416179401d14ad8"
server_key_version: "encryptionkey@0"
I20260810 07:20:07.964980 6195 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:07.967434 6220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:07.968807 6195 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:07.969009 6195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "3cf424168ff74c499e9d8b4344d174d3"
format_stamp: "Formatted at 2026-08-10 07:20:07 on dist-test-slave-44d2"
server_key: "8cf45f0e17940b2d0e727d22d80edcd7"
server_key_iv: "3a42cae8b4f0294a8416179401d14ad8"
server_key_version: "encryptionkey@0"
I20260810 07:20:07.969129 6195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:07.998912 6195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:08.002087 6195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:08.002310 6195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:08.010166 6195 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:35931
I20260810 07:20:08.010449 6306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:35931 every 8 connection(s)
I20260810 07:20:08.011219 6195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:08.014087 6309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:08.018903 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 6195
I20260810 07:20:08.019084 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:08.019372 1527 external_mini_cluster.cc:1428] Setting key a6de75243dbe210724585708f224f6fd
I20260810 07:20:08.019886 6309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3: Bootstrap starting.
I20260810 07:20:08.022727 6309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:08.023658 6309 log.cc:824] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:08.026113 6309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3: No bootstrap required, opened a new log
Aug 10 07:20:08 dist-test-slave-44d2 krb5kdc[6181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:08.030736 6309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf424168ff74c499e9d8b4344d174d3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 35931 } }
I20260810 07:20:08.031019 6309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:08.031103 6309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf424168ff74c499e9d8b4344d174d3, State: Initialized, Role: FOLLOWER
I20260810 07:20:08.031590 6309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf424168ff74c499e9d8b4344d174d3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 35931 } }
I20260810 07:20:08.031729 6309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:08.031836 6309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:08.031960 6309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:08.033241 6309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf424168ff74c499e9d8b4344d174d3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 35931 } }
I20260810 07:20:08.033619 6309 leader_election.cc:302] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cf424168ff74c499e9d8b4344d174d3; no voters:
I20260810 07:20:08.034019 6309 leader_election.cc:288] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:08.034145 6316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:08.034439 6316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [term 1 LEADER]: Becoming Leader. State: Replica: 3cf424168ff74c499e9d8b4344d174d3, State: Running, Role: LEADER
I20260810 07:20:08.034795 6316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf424168ff74c499e9d8b4344d174d3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 35931 } }
I20260810 07:20:08.035359 6309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:08.035941 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:08.020986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59322 (local address 127.1.125.254:35931)
0810 07:20:08.021522 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:08.021542 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:08.021589 (+ 47us) server_negotiation.cc:407] Connection header received
0810 07:20:08.022397 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:08.022423 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:08.022883 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:08.023266 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:08.024063 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.025421 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.026369 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.027048 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.027203 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.030139 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:08.030144 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:08.030158 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:08.030185 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:08.032547 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:08.033123 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:08.033132 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:08.033141 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:08.033302 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:08.033653 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:08.033656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:08.033657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:08.034114 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:08.034336 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:08.034652 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 07:20:08.034951 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":173,"threads_started":1}
I20260810 07:20:08.036722 6318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cf424168ff74c499e9d8b4344d174d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf424168ff74c499e9d8b4344d174d3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 35931 } } }
I20260810 07:20:08.037101 6318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:08.037431 6318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cf424168ff74c499e9d8b4344d174d3. Latest consensus state: current_term: 1 leader_uuid: "3cf424168ff74c499e9d8b4344d174d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf424168ff74c499e9d8b4344d174d3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 35931 } } }
I20260810 07:20:08.037612 6318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3 [sys.catalog]: This master's current role is: LEADER
W20260810 07:20:08.041384 6339 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:20:08.041474 6339 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:20:08.041597 6331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:08.042476 6331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:08.048472 6331 catalog_manager.cc:1347] Generated new cluster ID: 61904da753b64657a4ab1a745ff39d98
I20260810 07:20:08.048580 6331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:08.056445 6331 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:08.057490 6331 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:08.065440 6331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cf424168ff74c499e9d8b4344d174d3: Generated new TSK 0
I20260810 07:20:08.066156 6331 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:20:08.095903 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:35931
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:08.213259 6346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:08.213515 6346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:08.213557 6346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:08.217430 6346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:08.217483 6346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:08.217540 6346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:08.222676 6346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:35931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:08.223637 6346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:08.224637 6346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:08.234362 6354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:08.234315 6353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:08.234404 6346 server_base.cc:1034] running on GCE node
W20260810 07:20:08.234321 6357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:08.234987 6346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:08.235735 6346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:08.236930 6346 hybrid_clock.cc:648] HybridClock initialized: now 1786346408236878 us; error 62 us; skew 500 ppm
Aug 10 07:20:08 dist-test-slave-44d2 krb5kdc[6181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:08.239866 6346 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:20:08.240814 6346 webserver.cc:460] Webserver started at http://127.1.125.193:36335/ using document root <none> and password file <none>
I20260810 07:20:08.241436 6346 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:08.241544 6346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:08.241803 6346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:08.243834 6346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "20ede047fbd040999dadbc4207f28114"
format_stamp: "Formatted at 2026-08-10 07:20:08 on dist-test-slave-44d2"
server_key: "630e2f5c89e848c323beee5fc70cf6b9"
server_key_iv: "5019b4ceea31774c828569fbf7e0194e"
server_key_version: "encryptionkey@0"
I20260810 07:20:08.244410 6346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "20ede047fbd040999dadbc4207f28114"
format_stamp: "Formatted at 2026-08-10 07:20:08 on dist-test-slave-44d2"
server_key: "630e2f5c89e848c323beee5fc70cf6b9"
server_key_iv: "5019b4ceea31774c828569fbf7e0194e"
server_key_version: "encryptionkey@0"
I20260810 07:20:08.248142 6346 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:08.250645 6369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:08.251926 6346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:08.252081 6346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "20ede047fbd040999dadbc4207f28114"
format_stamp: "Formatted at 2026-08-10 07:20:08 on dist-test-slave-44d2"
server_key: "630e2f5c89e848c323beee5fc70cf6b9"
server_key_iv: "5019b4ceea31774c828569fbf7e0194e"
server_key_version: "encryptionkey@0"
I20260810 07:20:08.252202 6346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:08.266992 6346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:08.270005 6346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:08.270236 6346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:08.270890 6346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:08.271809 6346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:08.271884 6346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:08.271961 6346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:08.272006 6346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:08.281334 6346 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:39945
I20260810 07:20:08.281378 6543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:39945 every 8 connection(s)
I20260810 07:20:08.282284 6346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:08.282640 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 6346
I20260810 07:20:08.282792 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:08.283159 1527 external_mini_cluster.cc:1428] Setting key 49240576a3c262e90994c475ed26dc93
Aug 10 07:20:08 dist-test-slave-44d2 krb5kdc[6181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
I20260810 07:20:08.297046 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:08.284366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:42157 (local address 127.1.125.254:35931)
0810 07:20:08.284544 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:08.284549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:08.285539 (+ 990us) server_negotiation.cc:407] Connection header received
0810 07:20:08.286965 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:08.286969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:08.287031 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:08.287138 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:08.288343 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.288964 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.290272 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.290508 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.290641 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.293194 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:08.293198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:08.293201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:08.293235 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:08.294724 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:08.295392 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:08.295396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:08.295398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:08.295486 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:08.295847 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:08.295851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:08.295853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:08.296037 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:08.296153 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:08.296721 (+ 568us) server_negotiation.cc:299] Negotiation successful
0810 07:20:08.296855 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 07:20:08.298110 6546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:08.284881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:35931 (local address 127.1.125.193:42157)
0810 07:20:08.285390 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:08.285435 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:08.286758 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:08.287324 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:08.287331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:08.287702 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:08.288192 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:08.288205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.289112 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.289115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:08.290138 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:08.290149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.290696 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.290700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:08.290868 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.291490 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:08.291508 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:08.293017 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:08.294855 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:08.294861 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:08.294876 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:08.295267 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:08.295584 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:08.295587 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:08.295589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:08.295742 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:08.296157 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:08.296167 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:08.296420 (+ 253us) client_negotiation.cc:769] Sending connection context
0810 07:20:08.296674 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 07:20:08.297004 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":144,"threads_started":1}
I20260810 07:20:08.299630 6544 heartbeater.cc:347] Connected to a master server at 127.1.125.254:35931
I20260810 07:20:08.299889 6544 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:08.300395 6544 heartbeater.cc:511] Master 127.1.125.254:35931 requested a full tablet report, sending...
I20260810 07:20:08.302076 6252 ts_manager.cc:194] Registered new tserver with Master: 20ede047fbd040999dadbc4207f28114 (127.1.125.193:39945)
I20260810 07:20:08.303224 6252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:42157
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:20:08.313527 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:35931
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:08.427989 6552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:08.428254 6552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:08.428297 6552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:08.432433 6552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:08.432505 6552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:08.432650 6552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:08.438081 6552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:35931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:08.439114 6552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:08.440155 6552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:08.449820 6557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:08.449826 6560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:08.450008 6552 server_base.cc:1034] running on GCE node
W20260810 07:20:08.450189 6558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:08.450564 6552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:08.451258 6552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:08.452450 6552 hybrid_clock.cc:648] HybridClock initialized: now 1786346408452424 us; error 35 us; skew 500 ppm
Aug 10 07:20:08 dist-test-slave-44d2 krb5kdc[6181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:08.455303 6552 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:20:08.456151 6552 webserver.cc:460] Webserver started at http://127.1.125.194:34079/ using document root <none> and password file <none>
I20260810 07:20:08.456876 6552 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:08.456943 6552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:08.457190 6552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:08.459365 6552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "a8f991235312410cabe974a72a80fcd7"
format_stamp: "Formatted at 2026-08-10 07:20:08 on dist-test-slave-44d2"
server_key: "f2351013e03755a8c62f3c98a9ec6469"
server_key_iv: "9772836bf60efd355a3d3b69a6282a0c"
server_key_version: "encryptionkey@0"
I20260810 07:20:08.459941 6552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "a8f991235312410cabe974a72a80fcd7"
format_stamp: "Formatted at 2026-08-10 07:20:08 on dist-test-slave-44d2"
server_key: "f2351013e03755a8c62f3c98a9ec6469"
server_key_iv: "9772836bf60efd355a3d3b69a6282a0c"
server_key_version: "encryptionkey@0"
I20260810 07:20:08.463857 6552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260810 07:20:08.466384 6572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:08.467677 6552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:20:08.467828 6552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "a8f991235312410cabe974a72a80fcd7"
format_stamp: "Formatted at 2026-08-10 07:20:08 on dist-test-slave-44d2"
server_key: "f2351013e03755a8c62f3c98a9ec6469"
server_key_iv: "9772836bf60efd355a3d3b69a6282a0c"
server_key_version: "encryptionkey@0"
I20260810 07:20:08.467942 6552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:08.490088 6552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:08.493088 6552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:08.493304 6552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:08.493918 6552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:08.494786 6552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:08.494858 6552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:08.494930 6552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:08.494971 6552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:08.504380 6552 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:44047
I20260810 07:20:08.504415 6736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:44047 every 8 connection(s)
I20260810 07:20:08.505393 6552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:08.510155 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 6552
I20260810 07:20:08.510298 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:08.510619 1527 external_mini_cluster.cc:1428] Setting key d81f3a39ca1d7f82ec0516b283c64e43
Aug 10 07:20:08 dist-test-slave-44d2 krb5kdc[6181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:08.520064 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:08.507216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:33103 (local address 127.1.125.254:35931)
0810 07:20:08.507360 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:08.507364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:08.508242 (+ 878us) server_negotiation.cc:407] Connection header received
0810 07:20:08.509599 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:08.509602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:08.509650 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:08.509724 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:08.511265 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.511872 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.513163 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.513563 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.513687 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.516419 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:08.516422 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:08.516424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:08.516448 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:08.517961 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:08.518502 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:08.518508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:08.518512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:08.518595 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:08.518948 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:08.518951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:08.518953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:08.519142 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:08.519249 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:08.519747 (+ 498us) server_negotiation.cc:299] Negotiation successful
0810 07:20:08.519874 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 07:20:08.520951 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:08.507607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:35931 (local address 127.1.125.194:33103)
0810 07:20:08.508080 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:08.508125 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:08.509362 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:08.509924 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:08.509936 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:08.510489 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:08.511142 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:08.511159 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.512029 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.512033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:08.513028 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:08.513038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.513710 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.513713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:08.513860 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.514522 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:08.514539 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:08.516245 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:08.518106 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:08.518112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:08.518127 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:08.518393 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:08.518692 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:08.518695 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:08.518697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:08.518841 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:08.519250 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:08.519258 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:08.519507 (+ 249us) client_negotiation.cc:769] Sending connection context
0810 07:20:08.519698 (+ 191us) client_negotiation.cc:240] Negotiation successful
0810 07:20:08.520013 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":128,"threads_started":1}
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
I20260810 07:20:08.522153 6737 heartbeater.cc:347] Connected to a master server at 127.1.125.254:35931
I20260810 07:20:08.522403 6737 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:08.522950 6737 heartbeater.cc:511] Master 127.1.125.254:35931 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
I20260810 07:20:08.524784 6252 ts_manager.cc:194] Registered new tserver with Master: a8f991235312410cabe974a72a80fcd7 (127.1.125.194:44047)
I20260810 07:20:08.525579 6252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:33103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:20:08.541879 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:35931
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:08.661172 6744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:08.661458 6744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:08.661546 6744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:08.665728 6744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:08.665836 6744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:08.665977 6744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:08.671695 6744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:35931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:08.672890 6744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:08.673960 6744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:08.683785 6750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:08.683796 6757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:08.683825 6751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:08.683946 6744 server_base.cc:1034] running on GCE node
I20260810 07:20:08.684438 6744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:08.685163 6744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:08.686326 6744 hybrid_clock.cc:648] HybridClock initialized: now 1786346408686293 us; error 44 us; skew 500 ppm
Aug 10 07:20:08 dist-test-slave-44d2 krb5kdc[6181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:08.689146 6744 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:20:08.689990 6744 webserver.cc:460] Webserver started at http://127.1.125.195:35807/ using document root <none> and password file <none>
I20260810 07:20:08.690559 6744 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:08.690627 6744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:08.690882 6744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:08.693010 6744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "ed551abea1d9452392a6a92c31d1ea3b"
format_stamp: "Formatted at 2026-08-10 07:20:08 on dist-test-slave-44d2"
server_key: "3bd9fb7b528c39fc32c21e444847ab43"
server_key_iv: "dbbf9b147bfe50571280ac96a68fc959"
server_key_version: "encryptionkey@0"
I20260810 07:20:08.693662 6744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "ed551abea1d9452392a6a92c31d1ea3b"
format_stamp: "Formatted at 2026-08-10 07:20:08 on dist-test-slave-44d2"
server_key: "3bd9fb7b528c39fc32c21e444847ab43"
server_key_iv: "dbbf9b147bfe50571280ac96a68fc959"
server_key_version: "encryptionkey@0"
I20260810 07:20:08.697480 6744 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:20:08.699919 6771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:08.701202 6744 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:08.701349 6744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "ed551abea1d9452392a6a92c31d1ea3b"
format_stamp: "Formatted at 2026-08-10 07:20:08 on dist-test-slave-44d2"
server_key: "3bd9fb7b528c39fc32c21e444847ab43"
server_key_iv: "dbbf9b147bfe50571280ac96a68fc959"
server_key_version: "encryptionkey@0"
I20260810 07:20:08.701462 6744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:08.714815 6744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:08.717721 6744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:08.717929 6744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:08.718519 6744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:08.719445 6744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:08.719520 6744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:08.719589 6744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:08.719645 6744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:08.729238 6744 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:38453
I20260810 07:20:08.729282 6949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:38453 every 8 connection(s)
I20260810 07:20:08.730157 6744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:08.739472 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 6744
I20260810 07:20:08.739652 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:08.739975 1527 external_mini_cluster.cc:1428] Setting key 11f3d15178a613d618e8346e626d8169
Aug 10 07:20:08 dist-test-slave-44d2 krb5kdc[6181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:08.744390 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:08.732018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:34277 (local address 127.1.125.254:35931)
0810 07:20:08.732162 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:08.732166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:08.733085 (+ 919us) server_negotiation.cc:407] Connection header received
0810 07:20:08.734504 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:08.734507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:08.734568 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:08.734674 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:08.735876 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.736654 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.737941 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.738160 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.738245 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.740913 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:08.740918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:08.740921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:08.740951 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:08.742579 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:08.743049 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:08.743051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:08.743052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:08.743135 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:08.743429 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:08.743431 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:08.743432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:08.743601 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:08.743677 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:08.744165 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 07:20:08.744269 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:20:08.745415 6954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:08.732397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:35931 (local address 127.1.125.195:34277)
0810 07:20:08.732930 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:08.732973 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:08.734272 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:08.734849 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:08.734856 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:08.735217 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:08.735731 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:08.735747 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.736823 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.736829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:08.737824 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:08.737836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.738300 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.738303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:08.738447 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.738962 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:08.738978 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:08.740732 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:08.742680 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:08.742685 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:08.742695 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:08.742948 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:08.743216 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:08.743218 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:08.743219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:08.743350 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:08.743692 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:08.743696 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:08.743917 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 07:20:08.744103 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 07:20:08.744376 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":143,"threads_started":1}
I20260810 07:20:08.746856 6952 heartbeater.cc:347] Connected to a master server at 127.1.125.254:35931
I20260810 07:20:08.747107 6952 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:08.747622 6952 heartbeater.cc:511] Master 127.1.125.254:35931 requested a full tablet report, sending...
I20260810 07:20:08.748730 6252 ts_manager.cc:194] Registered new tserver with Master: ed551abea1d9452392a6a92c31d1ea3b (127.1.125.195:38453)
I20260810 07:20:08.749295 6252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:34277
I20260810 07:20:08.754112 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:20:08 dist-test-slave-44d2 krb5kdc[6181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346408, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 07:20:08 dist-test-slave-44d2 krb5kdc[6181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346408, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:08.782142 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:08.773315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59336 (local address 127.1.125.254:35931)
0810 07:20:08.773458 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:08.773465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:08.773678 (+ 213us) server_negotiation.cc:407] Connection header received
0810 07:20:08.773859 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:08.773863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:08.773918 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:08.774032 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:08.774792 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.775335 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.776209 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:08.776438 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:08.776633 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:08.779029 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:08.779033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:08.779035 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:08.779062 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:08.780301 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:08.780806 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:08.780810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:08.780811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:08.780899 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:08.781190 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:08.781194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:08.781195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:08.781437 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:08.781588 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:08.781836 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 07:20:08.781961 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 07:20:08.782568 6252 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59336
I20260810 07:20:08.783447 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 6346
I20260810 07:20:08.801141 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 6552
I20260810 07:20:08.814087 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 6744
I20260810 07:20:08.825608 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 6195
2026-08-10T07:20:08Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2409 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:08 dist-test-slave-44d2 krb5kdc[6967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:08 dist-test-slave-44d2 krb5kdc[6967](info): set up 2 sockets
Aug 10 07:20:08 dist-test-slave-44d2 krb5kdc[6967](info): commencing operation
krb5kdc: starting...
W20260810 07:20:10.078591 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:20:10 dist-test-slave-44d2 krb5kdc[6967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:20:10.211534 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:44613
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:44613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:10.328544 6982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:10.328845 6982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:10.328920 6982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:10.332990 6982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:10.333076 6982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:10.333114 6982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:10.333171 6982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:10.333213 6982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:10.338656 6982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:44613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:44613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:10.339756 6982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:10.340878 6982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:10.349520 6990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:10.349644 6991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:10.349529 6994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:10.349860 6982 server_base.cc:1034] running on GCE node
I20260810 07:20:10.350435 6982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:10.351621 6982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:10.352787 6982 hybrid_clock.cc:648] HybridClock initialized: now 1786346410352756 us; error 42 us; skew 500 ppm
Aug 10 07:20:10 dist-test-slave-44d2 krb5kdc[6967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:10.359522 6982 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:20:10.360450 6982 webserver.cc:460] Webserver started at http://127.1.125.254:41023/ using document root <none> and password file <none>
I20260810 07:20:10.361060 6982 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:10.361122 6982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:10.361294 6982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:10.363346 6982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "1c9f198316c248f4b9710f081c62f110"
format_stamp: "Formatted at 2026-08-10 07:20:10 on dist-test-slave-44d2"
server_key: "3d7ec03c6d77e19943c835db67e249ec"
server_key_iv: "923eb416d9a7707903e011b2f3fc2d91"
server_key_version: "encryptionkey@0"
I20260810 07:20:10.363883 6982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "1c9f198316c248f4b9710f081c62f110"
format_stamp: "Formatted at 2026-08-10 07:20:10 on dist-test-slave-44d2"
server_key: "3d7ec03c6d77e19943c835db67e249ec"
server_key_iv: "923eb416d9a7707903e011b2f3fc2d91"
server_key_version: "encryptionkey@0"
I20260810 07:20:10.368014 6982 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:20:10.371428 7006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:10.373276 6982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 07:20:10.373427 6982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "1c9f198316c248f4b9710f081c62f110"
format_stamp: "Formatted at 2026-08-10 07:20:10 on dist-test-slave-44d2"
server_key: "3d7ec03c6d77e19943c835db67e249ec"
server_key_iv: "923eb416d9a7707903e011b2f3fc2d91"
server_key_version: "encryptionkey@0"
I20260810 07:20:10.373569 6982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:10.391284 6982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:10.395651 6982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:10.395895 6982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:10.406363 6982 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:44613
I20260810 07:20:10.406394 7088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:44613 every 8 connection(s)
I20260810 07:20:10.407907 6982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:10.410936 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 6982
I20260810 07:20:10.411175 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:10.411726 1527 external_mini_cluster.cc:1428] Setting key 1754ea16475dcbb369e21ff14dc863c6
I20260810 07:20:10.411923 7090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:10.421060 7090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110: Bootstrap starting.
I20260810 07:20:10.424113 7090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110: Neither blocks nor log segments found. Creating new log.
Aug 10 07:20:10 dist-test-slave-44d2 krb5kdc[6967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:10.425168 7090 log.cc:824] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:10.427851 7090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110: No bootstrap required, opened a new log
I20260810 07:20:10.430959 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:10.413715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38450 (local address 127.1.125.254:44613)
0810 07:20:10.414240 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:10.414266 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:10.414322 (+ 56us) server_negotiation.cc:407] Connection header received
0810 07:20:10.415532 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:10.415564 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:10.416069 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:10.416590 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:10.417527 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.419443 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.420994 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.421852 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.422043 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:10.425913 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:10.425922 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:10.425941 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:10.425980 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:10.428106 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:10.428611 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:10.428617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:10.428624 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:10.428750 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:10.429060 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:10.429063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:10.429065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:10.429467 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:10.429700 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:10.429950 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 07:20:10.430248 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":125,"threads_started":1}
I20260810 07:20:10.431605 7090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9f198316c248f4b9710f081c62f110" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 44613 } }
I20260810 07:20:10.431844 7090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:10.431934 7090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c9f198316c248f4b9710f081c62f110, State: Initialized, Role: FOLLOWER
I20260810 07:20:10.432392 7090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9f198316c248f4b9710f081c62f110" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 44613 } }
I20260810 07:20:10.432580 7090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:10.432657 7090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:10.432755 7090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:10.433792 7090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9f198316c248f4b9710f081c62f110" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 44613 } }
I20260810 07:20:10.434091 7090 leader_election.cc:302] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c9f198316c248f4b9710f081c62f110; no voters:
I20260810 07:20:10.434353 7090 leader_election.cc:288] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:10.434480 7096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:10.434723 7096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [term 1 LEADER]: Becoming Leader. State: Replica: 1c9f198316c248f4b9710f081c62f110, State: Running, Role: LEADER
I20260810 07:20:10.435026 7096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9f198316c248f4b9710f081c62f110" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 44613 } }
I20260810 07:20:10.435673 7090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:10.436854 7100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c9f198316c248f4b9710f081c62f110. Latest consensus state: current_term: 1 leader_uuid: "1c9f198316c248f4b9710f081c62f110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9f198316c248f4b9710f081c62f110" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 44613 } } }
I20260810 07:20:10.436998 7100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:10.437282 7098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c9f198316c248f4b9710f081c62f110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9f198316c248f4b9710f081c62f110" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 44613 } } }
I20260810 07:20:10.437386 7098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:10.437515 7107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:10.440776 7107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:10.446535 7107 catalog_manager.cc:1347] Generated new cluster ID: 57407b55d54a43ea9fe476b7bf14970e
I20260810 07:20:10.446614 7107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:10.454308 7107 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:10.455184 7107 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:10.460242 7107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c9f198316c248f4b9710f081c62f110: Generated new TSK 0
I20260810 07:20:10.460949 7107 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:20:10.503270 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:44613
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:10.624471 7126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:10.624787 7126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:10.624866 7126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:10.628974 7126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:10.629055 7126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:10.629145 7126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:10.634552 7126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:44613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:10.635871 7126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:10.637176 7126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:10.647289 7135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:10.647454 7137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:10.647289 7131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:10.651646 7126 server_base.cc:1034] running on GCE node
I20260810 07:20:10.652118 7126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:10.652985 7126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:10.654186 7126 hybrid_clock.cc:648] HybridClock initialized: now 1786346410654175 us; error 32 us; skew 500 ppm
Aug 10 07:20:10 dist-test-slave-44d2 krb5kdc[6967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:10.657105 7126 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:20:10.657982 7126 webserver.cc:460] Webserver started at http://127.1.125.193:33041/ using document root <none> and password file <none>
I20260810 07:20:10.658506 7126 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:10.658563 7126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:10.658742 7126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:10.661020 7126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "dc10bd806ae9498393ec8e578d6ea619"
format_stamp: "Formatted at 2026-08-10 07:20:10 on dist-test-slave-44d2"
server_key: "bf40190585e5d15618f89344ebc54eee"
server_key_iv: "85e5d27d79292e60f291383e2de68f05"
server_key_version: "encryptionkey@0"
I20260810 07:20:10.661595 7126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "dc10bd806ae9498393ec8e578d6ea619"
format_stamp: "Formatted at 2026-08-10 07:20:10 on dist-test-slave-44d2"
server_key: "bf40190585e5d15618f89344ebc54eee"
server_key_iv: "85e5d27d79292e60f291383e2de68f05"
server_key_version: "encryptionkey@0"
I20260810 07:20:10.665619 7126 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 07:20:10.668054 7148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:10.669301 7126 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:10.669453 7126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "dc10bd806ae9498393ec8e578d6ea619"
format_stamp: "Formatted at 2026-08-10 07:20:10 on dist-test-slave-44d2"
server_key: "bf40190585e5d15618f89344ebc54eee"
server_key_iv: "85e5d27d79292e60f291383e2de68f05"
server_key_version: "encryptionkey@0"
I20260810 07:20:10.669561 7126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:10.681471 7126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:10.684657 7126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:10.684870 7126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:10.685500 7126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:10.686511 7126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:10.686571 7126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:10.686631 7126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:10.686661 7126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:10.696563 7126 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:46569
I20260810 07:20:10.696597 7338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:46569 every 8 connection(s)
I20260810 07:20:10.697510 7126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:10.700922 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 7126
I20260810 07:20:10.701061 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:10.701383 1527 external_mini_cluster.cc:1428] Setting key 956a332fafcffb7c32d2b96ec1ef64c4
Aug 10 07:20:10 dist-test-slave-44d2 krb5kdc[6967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:20:10.713934 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:10.699342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:54497 (local address 127.1.125.254:44613)
0810 07:20:10.699482 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:10.699487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:10.700336 (+ 849us) server_negotiation.cc:407] Connection header received
0810 07:20:10.701983 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:10.701986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:10.702047 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:10.702145 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:10.703449 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.704297 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.705885 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.706126 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.706210 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:10.709360 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:10.709365 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:10.709367 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:10.709398 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:10.711270 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:10.711969 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:10.711972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:10.711974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:10.712078 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:10.712607 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:10.712612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:10.712623 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:10.712868 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:10.712967 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:10.713570 (+ 603us) server_negotiation.cc:299] Negotiation successful
0810 07:20:10.713732 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:20:10.714568 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:10.699734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:44613 (local address 127.1.125.193:54497)
0810 07:20:10.700186 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:10.700231 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:10.701764 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:10.702301 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:10.702308 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:10.702790 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:10.703285 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:10.703299 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.704434 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.704436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:10.705744 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:10.705759 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.706290 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.706296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:10.706522 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:10.707213 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:10.707229 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:10.709124 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:10.711410 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:10.711416 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:10.711432 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:10.711835 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:10.712194 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:10.712200 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:10.712204 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:10.712385 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:10.712989 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:10.712994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:10.713241 (+ 247us) client_negotiation.cc:769] Sending connection context
0810 07:20:10.713401 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 07:20:10.713719 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
I20260810 07:20:10.716269 7341 heartbeater.cc:347] Connected to a master server at 127.1.125.254:44613
I20260810 07:20:10.716581 7341 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:10.717099 7341 heartbeater.cc:511] Master 127.1.125.254:44613 requested a full tablet report, sending...
I20260810 07:20:10.718935 7031 ts_manager.cc:194] Registered new tserver with Master: dc10bd806ae9498393ec8e578d6ea619 (127.1.125.193:46569)
I20260810 07:20:10.720233 7031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:54497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:20:10.731779 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:44613
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:10.848866 7349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:10.849143 7349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:10.849232 7349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:10.853400 7349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:10.853454 7349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:10.853544 7349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:10.858927 7349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:44613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:10.859947 7349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:10.861052 7349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:10.871282 7362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:10.871259 7359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:10.871253 7358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:10.871670 7349 server_base.cc:1034] running on GCE node
I20260810 07:20:10.872164 7349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:10.872952 7349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:10.874123 7349 hybrid_clock.cc:648] HybridClock initialized: now 1786346410874085 us; error 48 us; skew 500 ppm
Aug 10 07:20:10 dist-test-slave-44d2 krb5kdc[6967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:10.877082 7349 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:20:10.878026 7349 webserver.cc:460] Webserver started at http://127.1.125.194:35443/ using document root <none> and password file <none>
I20260810 07:20:10.878697 7349 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:10.878763 7349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:10.879007 7349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:10.881266 7349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "12736e2ba2fe432d844339ef22dda09a"
format_stamp: "Formatted at 2026-08-10 07:20:10 on dist-test-slave-44d2"
server_key: "2f3e17e074ff369c57cf0348ac24ba52"
server_key_iv: "685f6b633f28af53697bbcb852ca8579"
server_key_version: "encryptionkey@0"
I20260810 07:20:10.881840 7349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "12736e2ba2fe432d844339ef22dda09a"
format_stamp: "Formatted at 2026-08-10 07:20:10 on dist-test-slave-44d2"
server_key: "2f3e17e074ff369c57cf0348ac24ba52"
server_key_iv: "685f6b633f28af53697bbcb852ca8579"
server_key_version: "encryptionkey@0"
I20260810 07:20:10.885675 7349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 07:20:10.888134 7372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:10.889395 7349 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:20:10.889542 7349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "12736e2ba2fe432d844339ef22dda09a"
format_stamp: "Formatted at 2026-08-10 07:20:10 on dist-test-slave-44d2"
server_key: "2f3e17e074ff369c57cf0348ac24ba52"
server_key_iv: "685f6b633f28af53697bbcb852ca8579"
server_key_version: "encryptionkey@0"
I20260810 07:20:10.889659 7349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:10.914620 7349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:10.917685 7349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:10.917908 7349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:10.918536 7349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:10.919467 7349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:10.919543 7349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:10.919607 7349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:10.919657 7349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:10.929246 7349 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:32991
I20260810 07:20:10.929316 7549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:32991 every 8 connection(s)
I20260810 07:20:10.930424 7349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:10.939425 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 7349
I20260810 07:20:10.939582 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:10.939967 1527 external_mini_cluster.cc:1428] Setting key 05143dca5ed51cb67de52962860e9078
Aug 10 07:20:10 dist-test-slave-44d2 krb5kdc[6967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:10.945883 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:10.932538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:37357 (local address 127.1.125.254:44613)
0810 07:20:10.932697 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:10.932700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:10.933641 (+ 941us) server_negotiation.cc:407] Connection header received
0810 07:20:10.935072 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:10.935075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:10.935129 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:10.935205 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:10.936692 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.937323 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.938627 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.938840 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.938923 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:10.941941 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:10.941944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:10.941946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:10.941970 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:10.943461 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:10.944099 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:10.944105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:10.944107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:10.944212 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:10.944585 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:10.944590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:10.944595 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:10.944847 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:10.944957 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:10.945482 (+ 525us) server_negotiation.cc:299] Negotiation successful
0810 07:20:10.945626 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 07:20:10.946529 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:10.932985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:44613 (local address 127.1.125.194:37357)
0810 07:20:10.933464 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:10.933520 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:10.934836 (+ 1316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:10.935381 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:10.935388 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:10.935757 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:10.936537 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:10.936568 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.937484 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.937487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:10.938515 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:10.938526 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:10.938975 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:10.939001 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:10.939211 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:10.939925 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:10.939946 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:10.941765 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:10.943588 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:10.943596 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:10.943620 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:10.943963 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:10.944305 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:10.944307 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:10.944308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:10.944439 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:10.944951 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:10.944956 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:10.945216 (+ 260us) client_negotiation.cc:769] Sending connection context
0810 07:20:10.945367 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 07:20:10.945621 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":122,"threads_started":1}
I20260810 07:20:10.947825 7550 heartbeater.cc:347] Connected to a master server at 127.1.125.254:44613
I20260810 07:20:10.948100 7550 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:10.948746 7550 heartbeater.cc:511] Master 127.1.125.254:44613 requested a full tablet report, sending...
I20260810 07:20:10.950081 7031 ts_manager.cc:194] Registered new tserver with Master: 12736e2ba2fe432d844339ef22dda09a (127.1.125.194:32991)
I20260810 07:20:10.950896 7031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:37357
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:20:10.972397 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:44613
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:11.089306 7557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:11.089594 7557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:11.089668 7557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:11.093873 7557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:11.093977 7557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:11.094086 7557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:11.099334 7557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:44613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:11.100381 7557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:11.101465 7557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:11.111445 7565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:11.111549 7574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:11.111502 7567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:11.112015 7557 server_base.cc:1034] running on GCE node
I20260810 07:20:11.112413 7557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:11.113099 7557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:11.114264 7557 hybrid_clock.cc:648] HybridClock initialized: now 1786346411114250 us; error 26 us; skew 500 ppm
Aug 10 07:20:11 dist-test-slave-44d2 krb5kdc[6967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346411, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:11.117182 7557 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:20:11.118098 7557 webserver.cc:460] Webserver started at http://127.1.125.195:44837/ using document root <none> and password file <none>
I20260810 07:20:11.118712 7557 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:11.118811 7557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:11.119053 7557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:11.121213 7557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "0c36712d218f426b9344f0e45d5eda2a"
format_stamp: "Formatted at 2026-08-10 07:20:11 on dist-test-slave-44d2"
server_key: "83523444dc71b5b477e0b7a5036c2bb1"
server_key_iv: "29368863582a2b7ffa23b54085ba32dd"
server_key_version: "encryptionkey@0"
I20260810 07:20:11.121809 7557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "0c36712d218f426b9344f0e45d5eda2a"
format_stamp: "Formatted at 2026-08-10 07:20:11 on dist-test-slave-44d2"
server_key: "83523444dc71b5b477e0b7a5036c2bb1"
server_key_iv: "29368863582a2b7ffa23b54085ba32dd"
server_key_version: "encryptionkey@0"
I20260810 07:20:11.125633 7557 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:20:11.128046 7585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:11.129298 7557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:11.129446 7557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "0c36712d218f426b9344f0e45d5eda2a"
format_stamp: "Formatted at 2026-08-10 07:20:11 on dist-test-slave-44d2"
server_key: "83523444dc71b5b477e0b7a5036c2bb1"
server_key_iv: "29368863582a2b7ffa23b54085ba32dd"
server_key_version: "encryptionkey@0"
I20260810 07:20:11.129580 7557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:11.140866 7557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:11.143603 7557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:11.143805 7557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:11.144378 7557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:11.145295 7557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:11.145370 7557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:11.145439 7557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:11.145500 7557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:11.155460 7557 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:34801
I20260810 07:20:11.155500 7752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:34801 every 8 connection(s)
I20260810 07:20:11.156360 7557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:11.159513 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 7557
I20260810 07:20:11.159652 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:11.159962 1527 external_mini_cluster.cc:1428] Setting key a9781e6ef65b9f9e5dca9d8f2946019b
Aug 10 07:20:11 dist-test-slave-44d2 krb5kdc[6967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346411, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:11.171898 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:11.158432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:49453 (local address 127.1.125.254:44613)
0810 07:20:11.158588 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:11.158592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:11.159354 (+ 762us) server_negotiation.cc:407] Connection header received
0810 07:20:11.161354 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:11.161364 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:11.161436 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:11.161555 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:11.162846 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.163476 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.164839 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.165098 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.165225 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:11.168005 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:11.168009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:11.168011 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:11.168037 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:11.169648 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:11.170206 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:11.170210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:11.170212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:11.170299 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:11.170667 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:11.170671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:11.170673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:11.170948 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:11.171079 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:11.171586 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 07:20:11.171717 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:20:11.172976 7758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:11.158682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:44613 (local address 127.1.125.195:49453)
0810 07:20:11.159157 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:11.159205 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:11.161127 (+ 1922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:11.161748 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:11.161756 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:11.162143 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:11.162684 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:11.162699 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.163622 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.163627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:11.164689 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:11.164700 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.165272 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.165277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:11.165471 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:11.166184 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:11.166202 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:11.167797 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:11.169797 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:11.169803 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:11.169820 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:11.170092 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:11.170414 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:11.170417 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:11.170419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:11.170563 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:11.171078 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:11.171086 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:11.171339 (+ 253us) client_negotiation.cc:769] Sending connection context
0810 07:20:11.171545 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 07:20:11.171875 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":104,"threads_started":1}
I20260810 07:20:11.174635 7756 heartbeater.cc:347] Connected to a master server at 127.1.125.254:44613
I20260810 07:20:11.174938 7756 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:11.175482 7756 heartbeater.cc:511] Master 127.1.125.254:44613 requested a full tablet report, sending...
I20260810 07:20:11.176582 7030 ts_manager.cc:194] Registered new tserver with Master: 0c36712d218f426b9344f0e45d5eda2a (127.1.125.195:34801)
I20260810 07:20:11.177155 7030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:49453
I20260810 07:20:11.186437 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:20:11 dist-test-slave-44d2 krb5kdc[6967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:20:11 dist-test-slave-44d2 krb5kdc[6967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.193@KRBTEST.COM
I20260810 07:20:11.215508 7765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:11.205965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43606 (local address 127.1.125.193:46569)
0810 07:20:11.206260 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:11.206279 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:11.206314 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:20:11.206370 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:11.206375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:11.206587 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:11.206711 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:11.207434 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.208243 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.209205 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.209598 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.209684 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:11.212076 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:11.212086 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:11.212101 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:11.212125 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:11.213726 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:11.214127 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:11.214132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:11.214133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:11.214212 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:11.214466 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:11.214468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:11.214469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:11.214719 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:11.214893 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:11.215129 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 07:20:11.215337 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":94,"threads_started":1}
Aug 10 07:20:11 dist-test-slave-44d2 krb5kdc[6967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:11.226393 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:11.218432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38466 (local address 127.1.125.254:44613)
0810 07:20:11.218582 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:11.218589 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:11.218609 (+ 20us) server_negotiation.cc:407] Connection header received
0810 07:20:11.218778 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:11.218781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:11.218837 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:11.218923 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:11.219568 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.220097 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.220971 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:11.221204 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:11.221320 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:11.223647 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:11.223650 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:11.223652 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:11.223675 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:11.224957 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:11.225328 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:11.225330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:11.225331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:11.225423 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:11.225662 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:11.225664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:11.225665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:11.225872 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:11.225987 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:11.226178 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 07:20:11.226273 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 07:20:11.226981 7030 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38466
I20260810 07:20:11.227725 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 7126
I20260810 07:20:11.250695 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 7349
I20260810 07:20:11.263383 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 7557
I20260810 07:20:11.276212 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 6982
2026-08-10T07:20:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2446 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:11 dist-test-slave-44d2 krb5kdc[7773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:11 dist-test-slave-44d2 krb5kdc[7773](info): set up 2 sockets
Aug 10 07:20:11 dist-test-slave-44d2 krb5kdc[7773](info): commencing operation
krb5kdc: starting...
W20260810 07:20:12.559505 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:20:12 dist-test-slave-44d2 krb5kdc[7773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:20:12.693722 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:46825
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:46825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:12.809015 7797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:12.809366 7797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:12.809469 7797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:12.814081 7797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:12.814168 7797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:12.814204 7797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:12.814255 7797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:12.814293 7797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:12.819599 7797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:46825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:46825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:12.820775 7797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:12.821921 7797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:12.830466 7807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:12.830498 7809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:12.830523 7805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:12.830576 7797 server_base.cc:1034] running on GCE node
I20260810 07:20:12.831290 7797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:12.832463 7797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:12.833729 7797 hybrid_clock.cc:648] HybridClock initialized: now 1786346412833697 us; error 43 us; skew 500 ppm
Aug 10 07:20:12 dist-test-slave-44d2 krb5kdc[7773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:12.841104 7797 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:20:12.841823 7797 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:12.841889 7797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:12.842059 7797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:12.844219 7797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "364387f475b54dcba8e171706b5ff7c5"
format_stamp: "Formatted at 2026-08-10 07:20:12 on dist-test-slave-44d2"
server_key: "9ac25156f8dc4f54cd7f958a3c2ad9b1"
server_key_iv: "53bad08dae8b7d8a076cc7e40254390d"
server_key_version: "encryptionkey@0"
I20260810 07:20:12.844802 7797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "364387f475b54dcba8e171706b5ff7c5"
format_stamp: "Formatted at 2026-08-10 07:20:12 on dist-test-slave-44d2"
server_key: "9ac25156f8dc4f54cd7f958a3c2ad9b1"
server_key_iv: "53bad08dae8b7d8a076cc7e40254390d"
server_key_version: "encryptionkey@0"
I20260810 07:20:12.848426 7797 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260810 07:20:12.850863 7820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:12.852098 7797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:20:12.852226 7797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "364387f475b54dcba8e171706b5ff7c5"
format_stamp: "Formatted at 2026-08-10 07:20:12 on dist-test-slave-44d2"
server_key: "9ac25156f8dc4f54cd7f958a3c2ad9b1"
server_key_iv: "53bad08dae8b7d8a076cc7e40254390d"
server_key_version: "encryptionkey@0"
I20260810 07:20:12.852332 7797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:12.866314 7797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:12.869128 7797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:12.869331 7797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:12.876466 7797 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:46825
I20260810 07:20:12.876756 7891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:46825 every 8 connection(s)
I20260810 07:20:12.877166 7797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:12.879968 7893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:12.880955 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 7797
I20260810 07:20:12.881114 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:12.881397 1527 external_mini_cluster.cc:1428] Setting key b0e87b7cd2f6657ee755bfa01600f39b
I20260810 07:20:12.886269 7893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5: Bootstrap starting.
I20260810 07:20:12.888598 7893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:12.889325 7893 log.cc:824] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5: Log is configured to *not* fsync() on all Append() calls
Aug 10 07:20:12 dist-test-slave-44d2 krb5kdc[7773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:12.891647 7893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5: No bootstrap required, opened a new log
I20260810 07:20:12.895395 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:12.882837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44202 (local address 127.1.125.254:46825)
0810 07:20:12.883285 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:12.883301 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:12.883335 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:20:12.883964 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:12.883989 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:12.884309 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:12.884684 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:12.885483 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.886509 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.887370 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:12.887839 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:12.888025 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:12.890760 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:12.890765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:12.890779 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:12.890804 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:12.892543 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:12.893040 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:12.893044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:12.893049 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:12.893158 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:12.893456 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:12.893458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:12.893459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:12.893837 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:12.894003 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:12.894357 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 07:20:12.894667 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":143,"threads_started":1}
I20260810 07:20:12.895908 7893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "364387f475b54dcba8e171706b5ff7c5" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46825 } }
I20260810 07:20:12.896126 7893 raft_consensus.cc:374] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:12.896188 7893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 364387f475b54dcba8e171706b5ff7c5, State: Initialized, Role: FOLLOWER
I20260810 07:20:12.896756 7893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "364387f475b54dcba8e171706b5ff7c5" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46825 } }
I20260810 07:20:12.896926 7893 raft_consensus.cc:388] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:12.896997 7893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:12.897095 7893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:12.898061 7893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "364387f475b54dcba8e171706b5ff7c5" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46825 } }
I20260810 07:20:12.898360 7893 leader_election.cc:302] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 364387f475b54dcba8e171706b5ff7c5; no voters:
I20260810 07:20:12.898610 7893 leader_election.cc:288] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:12.898727 7900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:12.898965 7900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [term 1 LEADER]: Becoming Leader. State: Replica: 364387f475b54dcba8e171706b5ff7c5, State: Running, Role: LEADER
I20260810 07:20:12.899346 7900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "364387f475b54dcba8e171706b5ff7c5" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46825 } }
I20260810 07:20:12.899868 7893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:12.901276 7903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 364387f475b54dcba8e171706b5ff7c5. Latest consensus state: current_term: 1 leader_uuid: "364387f475b54dcba8e171706b5ff7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "364387f475b54dcba8e171706b5ff7c5" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46825 } } }
I20260810 07:20:12.901284 7901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "364387f475b54dcba8e171706b5ff7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "364387f475b54dcba8e171706b5ff7c5" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46825 } } }
I20260810 07:20:12.901427 7901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:12.901427 7903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:12.901921 7913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:12.904873 7913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:12.910629 7913 catalog_manager.cc:1347] Generated new cluster ID: 073509e690474ca58fb6f03b3ac79749
I20260810 07:20:12.910753 7913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:12.928182 7913 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:12.929155 7913 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:12.938473 7913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5: Generated new TSK 0
I20260810 07:20:12.939196 7913 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:20:12.971941 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46825
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:13.086570 7935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:13.086822 7935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:13.086866 7935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:13.090809 7935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:13.090884 7935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:13.090945 7935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:13.096293 7935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:13.097391 7935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:13.098446 7935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:13.108520 7946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:13.108518 7943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:13.108518 7944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:13.108656 7935 server_base.cc:1034] running on GCE node
I20260810 07:20:13.109216 7935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:13.109845 7935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:13.111011 7935 hybrid_clock.cc:648] HybridClock initialized: now 1786346413110979 us; error 42 us; skew 500 ppm
Aug 10 07:20:13 dist-test-slave-44d2 krb5kdc[7773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346413, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:13.113873 7935 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:20:13.114647 7935 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:13.114711 7935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:13.114931 7935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:13.117076 7935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "5d4f219e5cb04678b9493073385fc442"
format_stamp: "Formatted at 2026-08-10 07:20:13 on dist-test-slave-44d2"
server_key: "c16992497d55a18343978fd1cc2dbdd0"
server_key_iv: "6dae374de4ab15ee7159569e3a13fdbf"
server_key_version: "encryptionkey@0"
I20260810 07:20:13.117640 7935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "5d4f219e5cb04678b9493073385fc442"
format_stamp: "Formatted at 2026-08-10 07:20:13 on dist-test-slave-44d2"
server_key: "c16992497d55a18343978fd1cc2dbdd0"
server_key_iv: "6dae374de4ab15ee7159569e3a13fdbf"
server_key_version: "encryptionkey@0"
I20260810 07:20:13.121556 7935 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:20:13.123903 7957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:13.125440 7935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:20:13.125584 7935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "5d4f219e5cb04678b9493073385fc442"
format_stamp: "Formatted at 2026-08-10 07:20:13 on dist-test-slave-44d2"
server_key: "c16992497d55a18343978fd1cc2dbdd0"
server_key_iv: "6dae374de4ab15ee7159569e3a13fdbf"
server_key_version: "encryptionkey@0"
I20260810 07:20:13.125684 7935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:13.145915 7935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:13.148857 7935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:13.149065 7935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:13.149646 7935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:13.150511 7935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:13.150560 7935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:13.150628 7935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:13.150660 7935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:13.160081 7935 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:38961
I20260810 07:20:13.160113 8117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:38961 every 8 connection(s)
I20260810 07:20:13.160746 7935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:13.168773 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 7935
I20260810 07:20:13.168920 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:13.169246 1527 external_mini_cluster.cc:1428] Setting key eb43b863577f8ba969bda5fbe60797fa
Aug 10 07:20:13 dist-test-slave-44d2 krb5kdc[7773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346413, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:13.175211 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.162509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:49373 (local address 127.1.125.254:46825)
0810 07:20:13.162662 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:13.162666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:13.163516 (+ 850us) server_negotiation.cc:407] Connection header received
0810 07:20:13.164997 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:13.165000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:13.165059 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:13.165167 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:13.166392 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.166949 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.168101 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.168346 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.168457 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.171221 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:13.171232 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:13.171235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:13.171261 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:13.172994 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:13.173563 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:13.173568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:13.173570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:13.173660 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:13.174031 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:13.174036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:13.174038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:13.174232 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:13.174340 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:13.174875 (+ 535us) server_negotiation.cc:299] Negotiation successful
0810 07:20:13.175027 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:20:13.176057 8122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.162895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:46825 (local address 127.1.125.193:49373)
0810 07:20:13.163369 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:13.163413 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:13.164776 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:13.165352 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:13.165359 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:13.165719 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:13.166240 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.166255 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.167091 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.167094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.167962 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.167982 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.168548 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.168553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.168726 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.169411 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:13.169428 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:13.171049 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:13.173127 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:13.173134 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:13.173150 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:13.173443 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:13.173758 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:13.173761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:13.173763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:13.173925 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:13.174341 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:13.174350 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:13.174629 (+ 279us) client_negotiation.cc:769] Sending connection context
0810 07:20:13.174833 (+ 204us) client_negotiation.cc:240] Negotiation successful
0810 07:20:13.175177 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":126,"threads_started":1}
I20260810 07:20:13.177531 8120 heartbeater.cc:347] Connected to a master server at 127.1.125.254:46825
I20260810 07:20:13.177790 8120 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:13.178303 8120 heartbeater.cc:511] Master 127.1.125.254:46825 requested a full tablet report, sending...
I20260810 07:20:13.179893 7840 ts_manager.cc:194] Registered new tserver with Master: 5d4f219e5cb04678b9493073385fc442 (127.1.125.193:38961)
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:20:13.181248 7840 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:49373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:20:13.199290 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46825
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:13.319311 8125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:13.319557 8125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:13.319640 8125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:13.323577 8125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:13.323632 8125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:13.323691 8125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:13.328924 8125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:13.330020 8125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:13.331027 8125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:13.340647 8138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:13.340649 8136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:13.340735 8125 server_base.cc:1034] running on GCE node
W20260810 07:20:13.340669 8135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:13.341354 8125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:13.342005 8125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:13.343209 8125 hybrid_clock.cc:648] HybridClock initialized: now 1786346413343185 us; error 26 us; skew 500 ppm
Aug 10 07:20:13 dist-test-slave-44d2 krb5kdc[7773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346413, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:13.346145 8125 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:20:13.346886 8125 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:13.346984 8125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:13.347229 8125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:13.349484 8125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "c0cd23632b004b43978f0e8437473fc0"
format_stamp: "Formatted at 2026-08-10 07:20:13 on dist-test-slave-44d2"
server_key: "4763614025fccb167f01c34b9f96d1c5"
server_key_iv: "1d32d6a0503b0901bd4612b9283a43ad"
server_key_version: "encryptionkey@0"
I20260810 07:20:13.350080 8125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "c0cd23632b004b43978f0e8437473fc0"
format_stamp: "Formatted at 2026-08-10 07:20:13 on dist-test-slave-44d2"
server_key: "4763614025fccb167f01c34b9f96d1c5"
server_key_iv: "1d32d6a0503b0901bd4612b9283a43ad"
server_key_version: "encryptionkey@0"
I20260810 07:20:13.354040 8125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:13.356722 8146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:13.358084 8125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:13.358228 8125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "c0cd23632b004b43978f0e8437473fc0"
format_stamp: "Formatted at 2026-08-10 07:20:13 on dist-test-slave-44d2"
server_key: "4763614025fccb167f01c34b9f96d1c5"
server_key_iv: "1d32d6a0503b0901bd4612b9283a43ad"
server_key_version: "encryptionkey@0"
I20260810 07:20:13.358340 8125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:13.386754 8125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:13.390095 8125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:13.390326 8125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:13.390983 8125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:13.391921 8125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:13.391999 8125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:13.392076 8125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:13.392127 8125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:13.402379 8125 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:35405
I20260810 07:20:13.402427 8308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:35405 every 8 connection(s)
I20260810 07:20:13.403021 8125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:13.407011 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 8125
I20260810 07:20:13.407181 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:13.407498 1527 external_mini_cluster.cc:1428] Setting key 6d494b6a0fd6e13c552be961b5bcfbef
Aug 10 07:20:13 dist-test-slave-44d2 krb5kdc[7773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346413, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:13.417935 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.404896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:59335 (local address 127.1.125.254:46825)
0810 07:20:13.405047 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:13.405052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:13.406133 (+ 1081us) server_negotiation.cc:407] Connection header received
0810 07:20:13.407541 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:13.407544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:13.407602 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:13.407686 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:13.409100 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.409638 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.410789 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.411189 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.411309 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.414185 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:13.414188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:13.414191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:13.414217 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:13.415764 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:13.416310 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:13.416314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:13.416316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:13.416401 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:13.416783 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:13.416787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:13.416789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:13.416982 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:13.417097 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:13.417627 (+ 530us) server_negotiation.cc:299] Negotiation successful
0810 07:20:13.417756 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
I20260810 07:20:13.418792 8311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.405411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:46825 (local address 127.1.125.194:59335)
0810 07:20:13.405941 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:13.405989 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:13.407321 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:13.407889 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:13.407897 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:13.408336 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:13.408957 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.408979 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.409768 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.409771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.410656 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.410666 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.411352 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.411355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.411506 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.412143 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:13.412161 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:13.413985 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:13.415895 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:13.415901 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:13.415916 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:13.416198 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:13.416515 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:13.416519 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:13.416521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:13.416677 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:13.417106 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:13.417114 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:13.417364 (+ 250us) client_negotiation.cc:769] Sending connection context
0810 07:20:13.417555 (+ 191us) client_negotiation.cc:240] Negotiation successful
0810 07:20:13.417890 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":145,"threads_started":1}
I20260810 07:20:13.420011 8309 heartbeater.cc:347] Connected to a master server at 127.1.125.254:46825
I20260810 07:20:13.420295 8309 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:13.420837 8309 heartbeater.cc:511] Master 127.1.125.254:46825 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
I20260810 07:20:13.422093 7840 ts_manager.cc:194] Registered new tserver with Master: c0cd23632b004b43978f0e8437473fc0 (127.1.125.194:35405)
I20260810 07:20:13.422987 7840 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:59335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:20:13.439201 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46825
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:13.557647 8318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:13.557904 8318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:13.558039 8318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:13.562131 8318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:13.562188 8318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:13.562294 8318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:13.567721 8318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:13.568830 8318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:13.569845 8318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:13.579841 8328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:13.579841 8333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:13.580004 8329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:13.579927 8318 server_base.cc:1034] running on GCE node
I20260810 07:20:13.580586 8318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:13.581290 8318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:13.582486 8318 hybrid_clock.cc:648] HybridClock initialized: now 1786346413582447 us; error 45 us; skew 500 ppm
Aug 10 07:20:13 dist-test-slave-44d2 krb5kdc[7773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346413, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:13.585892 8318 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:20:13.586655 8318 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:13.586753 8318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:13.586998 8318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:13.589151 8318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "4fd81847ae2242d78870df351d06fc4e"
format_stamp: "Formatted at 2026-08-10 07:20:13 on dist-test-slave-44d2"
server_key: "68029735b116b89c763024031515cf03"
server_key_iv: "b318fdcb1f0a96b63d2d83ea15ca3645"
server_key_version: "encryptionkey@0"
I20260810 07:20:13.589722 8318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "4fd81847ae2242d78870df351d06fc4e"
format_stamp: "Formatted at 2026-08-10 07:20:13 on dist-test-slave-44d2"
server_key: "68029735b116b89c763024031515cf03"
server_key_iv: "b318fdcb1f0a96b63d2d83ea15ca3645"
server_key_version: "encryptionkey@0"
I20260810 07:20:13.593729 8318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:20:13.596338 8340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:13.597623 8318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:13.597781 8318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "4fd81847ae2242d78870df351d06fc4e"
format_stamp: "Formatted at 2026-08-10 07:20:13 on dist-test-slave-44d2"
server_key: "68029735b116b89c763024031515cf03"
server_key_iv: "b318fdcb1f0a96b63d2d83ea15ca3645"
server_key_version: "encryptionkey@0"
I20260810 07:20:13.597899 8318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:13.610373 8318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:13.613813 8318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:13.614027 8318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:13.614629 8318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:13.615489 8318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:13.615562 8318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:13.615629 8318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:13.615671 8318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:13.626011 8318 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:36345
I20260810 07:20:13.626008 8514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:36345 every 8 connection(s)
I20260810 07:20:13.626652 8318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:20:13 dist-test-slave-44d2 krb5kdc[7773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346413, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:13.636340 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 8318
I20260810 07:20:13.636484 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:13.636912 1527 external_mini_cluster.cc:1428] Setting key 4228bd1f9b3c92b65c1a0e293f3fe529
I20260810 07:20:13.640797 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.628461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:38615 (local address 127.1.125.254:46825)
0810 07:20:13.628670 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:13.628675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:13.629668 (+ 993us) server_negotiation.cc:407] Connection header received
0810 07:20:13.631075 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:13.631078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:13.631130 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:13.631207 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:13.632368 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.633006 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.634107 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.634316 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.634391 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.636932 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:13.636937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:13.636941 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:13.636972 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:13.638763 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:13.639293 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:13.639297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:13.639298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:13.639395 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:13.639714 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:13.639717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:13.639718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:13.639925 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:13.640039 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:13.640545 (+ 506us) server_negotiation.cc:299] Negotiation successful
0810 07:20:13.640671 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 07:20:13.641683 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.628947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:46825 (local address 127.1.125.195:38615)
0810 07:20:13.629514 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:13.629559 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:13.630869 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:13.631375 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:13.631382 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:13.631737 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:13.632239 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.632253 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.633121 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.633124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.634002 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.634011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.634443 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.634445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.634589 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.635111 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:13.635127 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:13.636742 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:13.638870 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:13.638875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:13.638886 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:13.639205 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:13.639492 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:13.639494 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:13.639496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:13.639631 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:13.640057 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:13.640062 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:13.640286 (+ 224us) client_negotiation.cc:769] Sending connection context
0810 07:20:13.640444 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 07:20:13.640789 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":122,"threads_started":1}
I20260810 07:20:13.642784 8518 heartbeater.cc:347] Connected to a master server at 127.1.125.254:46825
I20260810 07:20:13.643013 8518 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:13.643421 8518 heartbeater.cc:511] Master 127.1.125.254:46825 requested a full tablet report, sending...
I20260810 07:20:13.644412 7840 ts_manager.cc:194] Registered new tserver with Master: 4fd81847ae2242d78870df351d06fc4e (127.1.125.195:36345)
I20260810 07:20:13.645176 7840 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:38615
I20260810 07:20:13.650967 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:20:13.660257 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.653106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44218 (local address 127.1.125.254:46825)
0810 07:20:13.653223 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:13.653227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:13.653401 (+ 174us) server_negotiation.cc:407] Connection header received
0810 07:20:13.653498 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:13.653500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:13.653538 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:13.653614 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:13.654381 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.654882 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.655715 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.655961 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.656040 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.657524 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:13.657527 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:13.657528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:13.657550 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:13.658818 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:13.659210 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:13.659213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:13.659214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:13.659287 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:13.659510 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:13.659512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:13.659513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:13.659731 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:13.659832 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:13.660044 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 07:20:13.660138 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:20:13.663081 7840 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:20:13.666168 7840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:13.682286 8530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.675720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:35405 (local address 127.0.0.1:44772)
0810 07:20:13.676179 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:13.676192 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:13.676268 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:13.677003 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:13.677012 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:13.677037 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:13.677279 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.677287 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.679957 (+ 2670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.679960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.680911 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.680921 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.681714 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.681718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.681827 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.681868 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 07:20:13.681985 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 07:20:13.682111 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":68,"thread_start_us":93,"threads_started":1}
I20260810 07:20:13.682475 8534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.675274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42768 (local address 127.1.125.195:36345)
0810 07:20:13.675940 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:13.675958 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:13.676207 (+ 249us) server_negotiation.cc:407] Connection header received
0810 07:20:13.676457 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:13.676464 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:13.676707 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:13.676863 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:13.677417 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.678641 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.680744 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.681580 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.681689 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.681843 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:13.682181 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 07:20:13.682283 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":182,"threads_started":1}
I20260810 07:20:13.682691 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.675314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57178 (local address 127.1.125.193:38961)
0810 07:20:13.675766 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:13.675774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:13.676330 (+ 556us) server_negotiation.cc:407] Connection header received
0810 07:20:13.676426 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:13.676432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:13.676699 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:13.676855 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:13.677450 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.678637 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.681091 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.681838 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.681948 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.682093 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:13.682439 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 07:20:13.682538 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":124,"threads_started":1}
I20260810 07:20:13.682925 8536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.674952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:36345 (local address 127.0.0.1:42768)
0810 07:20:13.676105 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:13.676120 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:13.676295 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:13.677081 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:13.677084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:13.677100 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:13.677297 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.677302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.678770 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.678774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.680610 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.680622 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.681916 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.681920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.682009 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.682033 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:20:13.682693 (+ 660us) client_negotiation.cc:240] Negotiation successful
0810 07:20:13.682784 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"mutex_wait_us":224,"thread_start_us":103,"threads_started":1}
I20260810 07:20:13.682334 8532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.674916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:38961 (local address 127.0.0.1:57178)
0810 07:20:13.675954 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:13.676041 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:13.676179 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:13.677037 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:13.677039 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:13.677051 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:13.677302 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.677314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.678770 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.678774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.680610 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.680622 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.681947 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.681954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.682048 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.682070 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:20:13.682165 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 07:20:13.682224 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":291,"thread_start_us":98,"threads_started":1}
I20260810 07:20:13.683233 8537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.676150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.1.125.194:35405)
0810 07:20:13.676558 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:13.676571 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:13.676593 (+ 22us) server_negotiation.cc:407] Connection header received
0810 07:20:13.676662 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:13.676667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:13.676833 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:13.677175 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:13.678810 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.679822 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.681037 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.681573 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.681711 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.681855 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:13.682181 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 07:20:13.683076 (+ 895us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":109,"threads_started":1}
I20260810 07:20:13.685945 8023 tablet_service.cc:1467] Processing CreateTablet for tablet 5d6c1f3e259f4850b11e36b7ecbf1932 (DEFAULT_TABLE table=test-table [id=c53d368f97634c618d8a708c9fdf20db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:13.686071 8414 tablet_service.cc:1467] Processing CreateTablet for tablet 5d6c1f3e259f4850b11e36b7ecbf1932 (DEFAULT_TABLE table=test-table [id=c53d368f97634c618d8a708c9fdf20db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:13.686087 8212 tablet_service.cc:1467] Processing CreateTablet for tablet 5d6c1f3e259f4850b11e36b7ecbf1932 (DEFAULT_TABLE table=test-table [id=c53d368f97634c618d8a708c9fdf20db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:13.687119 8023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d6c1f3e259f4850b11e36b7ecbf1932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:13.687136 8212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d6c1f3e259f4850b11e36b7ecbf1932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:13.687158 8414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d6c1f3e259f4850b11e36b7ecbf1932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:13.693476 8538 tablet_bootstrap.cc:492] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442: Bootstrap starting.
I20260810 07:20:13.695017 8542 tablet_bootstrap.cc:492] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0: Bootstrap starting.
I20260810 07:20:13.695827 8538 tablet_bootstrap.cc:654] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:13.696794 8538 log.cc:824] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:13.697068 8542 tablet_bootstrap.cc:654] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:13.697537 8539 tablet_bootstrap.cc:492] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e: Bootstrap starting.
I20260810 07:20:13.698278 8542 log.cc:824] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:13.699429 8539 tablet_bootstrap.cc:654] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:13.699967 8538 tablet_bootstrap.cc:492] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442: No bootstrap required, opened a new log
I20260810 07:20:13.700186 8538 ts_tablet_manager.cc:1397] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260810 07:20:13.700310 8542 tablet_bootstrap.cc:492] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0: No bootstrap required, opened a new log
I20260810 07:20:13.700528 8542 ts_tablet_manager.cc:1397] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:20:13.700933 8539 log.cc:824] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:13.702772 8539 tablet_bootstrap.cc:492] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e: No bootstrap required, opened a new log
I20260810 07:20:13.702997 8539 ts_tablet_manager.cc:1397] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:20:13.704651 8542 raft_consensus.cc:348] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4f219e5cb04678b9493073385fc442" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 38961 } } peers { permanent_uuid: "4fd81847ae2242d78870df351d06fc4e" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36345 } } peers { permanent_uuid: "c0cd23632b004b43978f0e8437473fc0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35405 } }
I20260810 07:20:13.704962 8542 raft_consensus.cc:374] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:13.705053 8542 raft_consensus.cc:729] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0cd23632b004b43978f0e8437473fc0, State: Initialized, Role: FOLLOWER
I20260810 07:20:13.704872 8538 raft_consensus.cc:348] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4f219e5cb04678b9493073385fc442" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 38961 } } peers { permanent_uuid: "4fd81847ae2242d78870df351d06fc4e" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36345 } } peers { permanent_uuid: "c0cd23632b004b43978f0e8437473fc0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35405 } }
I20260810 07:20:13.705317 8538 raft_consensus.cc:374] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:13.705417 8538 raft_consensus.cc:729] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4f219e5cb04678b9493073385fc442, State: Initialized, Role: FOLLOWER
I20260810 07:20:13.705610 8542 consensus_queue.cc:260] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4f219e5cb04678b9493073385fc442" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 38961 } } peers { permanent_uuid: "4fd81847ae2242d78870df351d06fc4e" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36345 } } peers { permanent_uuid: "c0cd23632b004b43978f0e8437473fc0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35405 } }
I20260810 07:20:13.705843 8538 consensus_queue.cc:260] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4f219e5cb04678b9493073385fc442" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 38961 } } peers { permanent_uuid: "4fd81847ae2242d78870df351d06fc4e" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36345 } } peers { permanent_uuid: "c0cd23632b004b43978f0e8437473fc0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35405 } }
I20260810 07:20:13.706331 8309 heartbeater.cc:503] Master 127.1.125.254:46825 was elected leader, sending a full tablet report...
I20260810 07:20:13.706667 8542 ts_tablet_manager.cc:1428] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:20:13.706907 8538 ts_tablet_manager.cc:1428] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260810 07:20:13.706854 8539 raft_consensus.cc:348] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4f219e5cb04678b9493073385fc442" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 38961 } } peers { permanent_uuid: "4fd81847ae2242d78870df351d06fc4e" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36345 } } peers { permanent_uuid: "c0cd23632b004b43978f0e8437473fc0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35405 } }
I20260810 07:20:13.707248 8539 raft_consensus.cc:374] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:13.707265 8120 heartbeater.cc:503] Master 127.1.125.254:46825 was elected leader, sending a full tablet report...
I20260810 07:20:13.707374 8539 raft_consensus.cc:729] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fd81847ae2242d78870df351d06fc4e, State: Initialized, Role: FOLLOWER
I20260810 07:20:13.707842 8539 consensus_queue.cc:260] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4f219e5cb04678b9493073385fc442" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 38961 } } peers { permanent_uuid: "4fd81847ae2242d78870df351d06fc4e" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36345 } } peers { permanent_uuid: "c0cd23632b004b43978f0e8437473fc0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35405 } }
I20260810 07:20:13.708495 8518 heartbeater.cc:503] Master 127.1.125.254:46825 was elected leader, sending a full tablet report...
I20260810 07:20:13.709794 8539 ts_tablet_manager.cc:1428] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260810 07:20:13.853503 8553 raft_consensus.cc:482] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:13.853798 8553 raft_consensus.cc:504] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4f219e5cb04678b9493073385fc442" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 38961 } } peers { permanent_uuid: "4fd81847ae2242d78870df351d06fc4e" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36345 } } peers { permanent_uuid: "c0cd23632b004b43978f0e8437473fc0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35405 } }
I20260810 07:20:13.855106 8553 leader_election.cc:288] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d4f219e5cb04678b9493073385fc442 (127.1.125.193:38961), c0cd23632b004b43978f0e8437473fc0 (127.1.125.194:35405)
I20260810 07:20:13.859629 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.855514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:38961 (local address 127.1.125.195:56775)
0810 07:20:13.855690 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:13.855708 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:13.855835 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:13.856308 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:13.856312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:13.856334 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:13.856450 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.856455 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.857391 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.857394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.858358 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.858368 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.859164 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.859168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.859280 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.859307 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 07:20:13.859405 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 07:20:13.859481 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 07:20:13.859685 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.855594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:56775 (local address 127.1.125.193:38961)
0810 07:20:13.855738 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:13.855745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:13.855767 (+ 22us) server_negotiation.cc:407] Connection header received
0810 07:20:13.855981 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:13.855985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:13.856055 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:13.856144 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:13.856607 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.857274 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.858512 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.859030 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.859125 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.859167 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:13.859445 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 07:20:13.859540 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:20:13.859879 8554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.855835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:35405 (local address 127.1.125.195:34843)
0810 07:20:13.856313 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:13.856327 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:13.856455 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:13.856767 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:13.856769 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:13.856789 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:13.856922 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.856929 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.857788 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.857791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.858695 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:13.858708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.859458 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.859462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:13.859564 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.859587 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:20:13.859666 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:20:13.859740 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":177,"threads_started":1}
I20260810 07:20:13.859941 8537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.856039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:34843 (local address 127.1.125.194:35405)
0810 07:20:13.856207 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:13.856212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:13.856418 (+ 206us) server_negotiation.cc:407] Connection header received
0810 07:20:13.856608 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:13.856612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:13.856667 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:13.856728 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:13.857046 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.857672 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.858827 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.859320 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.859444 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.859490 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:13.859707 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 07:20:13.859795 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 07:20:13.860363 8048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d6c1f3e259f4850b11e36b7ecbf1932" candidate_uuid: "4fd81847ae2242d78870df351d06fc4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4f219e5cb04678b9493073385fc442" is_pre_election: true
I20260810 07:20:13.860725 8239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d6c1f3e259f4850b11e36b7ecbf1932" candidate_uuid: "4fd81847ae2242d78870df351d06fc4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0cd23632b004b43978f0e8437473fc0" is_pre_election: true
I20260810 07:20:13.860775 8048 raft_consensus.cc:2454] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fd81847ae2242d78870df351d06fc4e in term 0.
I20260810 07:20:13.861176 8239 raft_consensus.cc:2454] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fd81847ae2242d78870df351d06fc4e in term 0.
I20260810 07:20:13.861420 8347 leader_election.cc:302] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fd81847ae2242d78870df351d06fc4e, 5d4f219e5cb04678b9493073385fc442; no voters:
I20260810 07:20:13.861783 8553 raft_consensus.cc:2781] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:13.861946 8553 raft_consensus.cc:482] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:13.862031 8553 raft_consensus.cc:3037] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:13.863263 8553 raft_consensus.cc:504] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4f219e5cb04678b9493073385fc442" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 38961 } } peers { permanent_uuid: "4fd81847ae2242d78870df351d06fc4e" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36345 } } peers { permanent_uuid: "c0cd23632b004b43978f0e8437473fc0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35405 } }
I20260810 07:20:13.863629 8553 leader_election.cc:288] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [CANDIDATE]: Term 1 election: Requested vote from peers 5d4f219e5cb04678b9493073385fc442 (127.1.125.193:38961), c0cd23632b004b43978f0e8437473fc0 (127.1.125.194:35405)
I20260810 07:20:13.864028 8048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d6c1f3e259f4850b11e36b7ecbf1932" candidate_uuid: "4fd81847ae2242d78870df351d06fc4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4f219e5cb04678b9493073385fc442"
I20260810 07:20:13.864151 8239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d6c1f3e259f4850b11e36b7ecbf1932" candidate_uuid: "4fd81847ae2242d78870df351d06fc4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0cd23632b004b43978f0e8437473fc0"
I20260810 07:20:13.864171 8048 raft_consensus.cc:3037] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:13.864272 8239 raft_consensus.cc:3037] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:13.865640 8239 raft_consensus.cc:2454] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fd81847ae2242d78870df351d06fc4e in term 1.
I20260810 07:20:13.865641 8048 raft_consensus.cc:2454] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fd81847ae2242d78870df351d06fc4e in term 1.
I20260810 07:20:13.865967 8347 leader_election.cc:302] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fd81847ae2242d78870df351d06fc4e, c0cd23632b004b43978f0e8437473fc0; no voters:
I20260810 07:20:13.866184 8553 raft_consensus.cc:2781] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:13.866479 8553 raft_consensus.cc:686] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [term 1 LEADER]: Becoming Leader. State: Replica: 4fd81847ae2242d78870df351d06fc4e, State: Running, Role: LEADER
I20260810 07:20:13.866892 8553 consensus_queue.cc:237] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4f219e5cb04678b9493073385fc442" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 38961 } } peers { permanent_uuid: "4fd81847ae2242d78870df351d06fc4e" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36345 } } peers { permanent_uuid: "c0cd23632b004b43978f0e8437473fc0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35405 } }
I20260810 07:20:13.870429 7840 catalog_manager.cc:5495] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4fd81847ae2242d78870df351d06fc4e (127.1.125.195). New cstate: current_term: 1 leader_uuid: "4fd81847ae2242d78870df351d06fc4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4f219e5cb04678b9493073385fc442" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 38961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fd81847ae2242d78870df351d06fc4e" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0cd23632b004b43978f0e8437473fc0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35405 } health_report { overall_health: UNKNOWN } } }
W20260810 07:20:13.877589 8519 tablet.cc:2369] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:20:13.904212 8310 tablet.cc:2369] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:20:13.912189 8121 tablet.cc:2369] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:20:13.915947 8534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.912192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42770 (local address 127.1.125.195:36345)
0810 07:20:13.912359 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:13.912367 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:13.912384 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:20:13.912531 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:13.912536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:13.912619 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:13.912744 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:13.913014 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.913535 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.914422 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:13.914657 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:13.914774 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:13.915099 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:13.915508 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:13.915656 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:13.915719 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 07:20:13.915779 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 07:20:13.921983 8239 raft_consensus.cc:1261] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0 [term 1 FOLLOWER]: Refusing update from remote peer 4fd81847ae2242d78870df351d06fc4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:13.922003 8048 raft_consensus.cc:1261] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442 [term 1 FOLLOWER]: Refusing update from remote peer 4fd81847ae2242d78870df351d06fc4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:13.922770 8555 consensus_queue.cc:1035] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0cd23632b004b43978f0e8437473fc0" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:13.922947 8553 consensus_queue.cc:1035] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4f219e5cb04678b9493073385fc442" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 38961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:13.929738 8560 mvcc.cc:205] Tried to move back new op lower bound from 7316874911416545280 to 7316874911202054144. Current Snapshot: MvccSnapshot[applied={T|T < 7316874911416545280}]
I20260810 07:20:13.932624 8561 mvcc.cc:205] Tried to move back new op lower bound from 7316874911416545280 to 7316874911202054144. Current Snapshot: MvccSnapshot[applied={T|T < 7316874911416545280}]
I20260810 07:20:13.939184 7839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:13.939399 7839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:13.942034 7839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 364387f475b54dcba8e171706b5ff7c5: Sending DeleteTablet for 3 replicas of tablet 5d6c1f3e259f4850b11e36b7ecbf1932
I20260810 07:20:13.942909 8414 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d6c1f3e259f4850b11e36b7ecbf1932 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:13 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:42768
I20260810 07:20:13.943008 8212 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d6c1f3e259f4850b11e36b7ecbf1932 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:13 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:44772
I20260810 07:20:13.943113 8023 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d6c1f3e259f4850b11e36b7ecbf1932 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:13 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:57178
I20260810 07:20:13.943578 8577 tablet_replica.cc:330] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442: stopping tablet replica
I20260810 07:20:13.943612 8574 tablet_replica.cc:330] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e: stopping tablet replica
I20260810 07:20:13.943966 8577 raft_consensus.cc:2229] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 5d4f219e5cb04678b9493073385fc442 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:13.943953 8574 raft_consensus.cc:2229] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:13.944005 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 7935
I20260810 07:20:13.944352 8574 raft_consensus.cc:2258] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:13.944845 8576 tablet_replica.cc:330] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0: stopping tablet replica
I20260810 07:20:13.945405 8576 raft_consensus.cc:2229] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:13.945788 8576 raft_consensus.cc:2258] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:13.951525 8574 ts_tablet_manager.cc:1905] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:13.951577 8576 ts_tablet_manager.cc:1905] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:13.954660 8574 ts_tablet_manager.cc:1918] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:13.954819 8574 log.cc:1196] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/5d6c1f3e259f4850b11e36b7ecbf1932
I20260810 07:20:13.955229 8574 ts_tablet_manager.cc:1939] T 5d6c1f3e259f4850b11e36b7ecbf1932 P 4fd81847ae2242d78870df351d06fc4e: Deleting consensus metadata
I20260810 07:20:13.955355 8576 ts_tablet_manager.cc:1918] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:13.955523 8576 log.cc:1196] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/5d6c1f3e259f4850b11e36b7ecbf1932
I20260810 07:20:13.956009 8576 ts_tablet_manager.cc:1939] T 5d6c1f3e259f4850b11e36b7ecbf1932 P c0cd23632b004b43978f0e8437473fc0: Deleting consensus metadata
I20260810 07:20:13.956718 7824 catalog_manager.cc:4841] TS 4fd81847ae2242d78870df351d06fc4e (127.1.125.195:36345): tablet 5d6c1f3e259f4850b11e36b7ecbf1932 (table test-table [id=c53d368f97634c618d8a708c9fdf20db]) successfully deleted
I20260810 07:20:13.957568 7825 catalog_manager.cc:4841] TS c0cd23632b004b43978f0e8437473fc0 (127.1.125.194:35405): tablet 5d6c1f3e259f4850b11e36b7ecbf1932 (table test-table [id=c53d368f97634c618d8a708c9fdf20db]) successfully deleted
W20260810 07:20:13.958606 7825 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:38961 (error 108)
I20260810 07:20:13.959661 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 8125
I20260810 07:20:13.959872 8532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:13.959494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:38961 (local address 127.0.0.1:57180)
0810 07:20:13.959666 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:13.959762 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260810 07:20:13.964123 7825 catalog_manager.cc:4584] TS 5d4f219e5cb04678b9493073385fc442 (127.1.125.193:38961): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5d6c1f3e259f4850b11e36b7ecbf1932: Network error: Client connection negotiation failed: client connection to 127.1.125.193:38961: connect: Connection refused (error 111)
I20260810 07:20:13.976590 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 8318
I20260810 07:20:13.989193 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 7797
2026-08-10T07:20:13Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2713 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T07:20:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:14Z Disabled control of system clock
I20260810 07:20:14.028784 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:39603
--webserver_interface=127.1.125.254
--webserver_port=0
--builtin_ntp_servers=127.1.125.212:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:39603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 07:20:14.150688 8585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:14.150965 8585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:14.151019 8585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:14.155005 8585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:14.155057 8585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:14.155073 8585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:14.155089 8585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:14.155105 8585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:14.159749 8585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:39603
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:39603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:14.160785 8585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:14.162101 8585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:14.170915 8592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:14.171002 8585 server_base.cc:1034] running on GCE node
W20260810 07:20:14.170923 8593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:14.170923 8601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:14.171799 8585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:14.172927 8585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:14.174131 8585 hybrid_clock.cc:648] HybridClock initialized: now 1786346414174098 us; error 44 us; skew 500 ppm
I20260810 07:20:14.176038 8585 webserver.cc:460] Webserver started at http://127.1.125.254:37497/ using document root <none> and password file <none>
I20260810 07:20:14.176702 8585 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:14.176834 8585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:14.177083 8585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:14.179139 8585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "67cfaa0e65654a01a61e60676923cd1d"
format_stamp: "Formatted at 2026-08-10 07:20:14 on dist-test-slave-44d2"
server_key: "62e31bf760511a0ad210c1851d0bd510"
server_key_iv: "e2bfc7df5e69ec8c61de4a8b1620043d"
server_key_version: "encryptionkey@0"
I20260810 07:20:14.179703 8585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "67cfaa0e65654a01a61e60676923cd1d"
format_stamp: "Formatted at 2026-08-10 07:20:14 on dist-test-slave-44d2"
server_key: "62e31bf760511a0ad210c1851d0bd510"
server_key_iv: "e2bfc7df5e69ec8c61de4a8b1620043d"
server_key_version: "encryptionkey@0"
I20260810 07:20:14.183907 8585 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:20:14.186604 8609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:14.187809 8585 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:14.188019 8585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "67cfaa0e65654a01a61e60676923cd1d"
format_stamp: "Formatted at 2026-08-10 07:20:14 on dist-test-slave-44d2"
server_key: "62e31bf760511a0ad210c1851d0bd510"
server_key_iv: "e2bfc7df5e69ec8c61de4a8b1620043d"
server_key_version: "encryptionkey@0"
I20260810 07:20:14.188145 8585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:14.192945 8585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:14.193545 8585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:14.193729 8585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:14.200876 8585 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:39603
I20260810 07:20:14.200886 8682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:39603 every 8 connection(s)
I20260810 07:20:14.201807 8585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:14.205050 8683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:14.205165 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 8585
I20260810 07:20:14.205324 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:14.205653 1527 external_mini_cluster.cc:1428] Setting key 48c931dd4a7b3020f83aebaf3721ff3a
I20260810 07:20:14.211540 8683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d: Bootstrap starting.
I20260810 07:20:14.211879 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.207180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33956 (local address 127.1.125.254:39603)
0810 07:20:14.207832 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.207852 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.207888 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:20:14.208646 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.208666 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.209022 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.209313 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.210443 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:14.210448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:14.210460 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:14.210472 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:14.210534 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:14.210552 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:14.210675 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:14.210819 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 07:20:14.211133 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":144,"threads_started":1}
I20260810 07:20:14.214186 8683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:14.214980 8683 log.cc:824] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:14.217123 8683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d: No bootstrap required, opened a new log
I20260810 07:20:14.220871 8683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67cfaa0e65654a01a61e60676923cd1d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39603 } }
I20260810 07:20:14.221131 8683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:14.221212 8683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67cfaa0e65654a01a61e60676923cd1d, State: Initialized, Role: FOLLOWER
I20260810 07:20:14.221683 8683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67cfaa0e65654a01a61e60676923cd1d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39603 } }
I20260810 07:20:14.221840 8683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:14.221930 8683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:14.222031 8683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:14.222942 8683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67cfaa0e65654a01a61e60676923cd1d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39603 } }
I20260810 07:20:14.223291 8683 leader_election.cc:302] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67cfaa0e65654a01a61e60676923cd1d; no voters:
I20260810 07:20:14.223613 8683 leader_election.cc:288] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:14.223768 8695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:14.224030 8695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [term 1 LEADER]: Becoming Leader. State: Replica: 67cfaa0e65654a01a61e60676923cd1d, State: Running, Role: LEADER
I20260810 07:20:14.224382 8695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67cfaa0e65654a01a61e60676923cd1d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39603 } }
I20260810 07:20:14.224901 8683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:14.226174 8696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67cfaa0e65654a01a61e60676923cd1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67cfaa0e65654a01a61e60676923cd1d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39603 } } }
I20260810 07:20:14.226325 8696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:14.226155 8697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67cfaa0e65654a01a61e60676923cd1d. Latest consensus state: current_term: 1 leader_uuid: "67cfaa0e65654a01a61e60676923cd1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67cfaa0e65654a01a61e60676923cd1d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39603 } } }
I20260810 07:20:14.226856 8697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d [sys.catalog]: This master's current role is: LEADER
W20260810 07:20:14.230043 8714 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:20:14.230122 8714 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:20:14.230209 8707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:14.230944 8707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:14.236299 8707 catalog_manager.cc:1347] Generated new cluster ID: 949bfdeb47514683adbf1b84e3a45720
I20260810 07:20:14.236367 8707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:14.249784 8707 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:14.250142 8707 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:14.260609 8707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d: Generated new TSK 0
I20260810 07:20:14.261329 8707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:20:14.272814 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39603
--builtin_ntp_servers=127.1.125.212:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 07:20:14.390254 8723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:14.390508 8723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:14.390547 8723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:14.394877 8723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:14.394932 8723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:14.395022 8723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:14.399878 8723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:14.401017 8723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:14.402431 8723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:14.412110 8734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:14.412130 8730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:14.412279 8723 server_base.cc:1034] running on GCE node
W20260810 07:20:14.412139 8732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:14.413087 8723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:14.413720 8723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:14.414911 8723 hybrid_clock.cc:648] HybridClock initialized: now 1786346414414876 us; error 47 us; skew 500 ppm
I20260810 07:20:14.416836 8723 webserver.cc:460] Webserver started at http://127.1.125.193:36027/ using document root <none> and password file <none>
I20260810 07:20:14.417420 8723 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:14.417523 8723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:14.417762 8723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:14.419891 8723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "41c1f1caba084786bff3cc4ecdbd03db"
format_stamp: "Formatted at 2026-08-10 07:20:14 on dist-test-slave-44d2"
server_key: "6fbb8da5e52a1c43c4bf475ebcf1b688"
server_key_iv: "241cd20a8e8061f5aeb150ca3c0d355c"
server_key_version: "encryptionkey@0"
I20260810 07:20:14.420467 8723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "41c1f1caba084786bff3cc4ecdbd03db"
format_stamp: "Formatted at 2026-08-10 07:20:14 on dist-test-slave-44d2"
server_key: "6fbb8da5e52a1c43c4bf475ebcf1b688"
server_key_iv: "241cd20a8e8061f5aeb150ca3c0d355c"
server_key_version: "encryptionkey@0"
I20260810 07:20:14.424052 8723 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:20:14.426733 8743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:14.427893 8723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:14.428051 8723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "41c1f1caba084786bff3cc4ecdbd03db"
format_stamp: "Formatted at 2026-08-10 07:20:14 on dist-test-slave-44d2"
server_key: "6fbb8da5e52a1c43c4bf475ebcf1b688"
server_key_iv: "241cd20a8e8061f5aeb150ca3c0d355c"
server_key_version: "encryptionkey@0"
I20260810 07:20:14.428165 8723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:14.432713 8723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:14.433310 8723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:14.433501 8723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:14.434047 8723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:14.434950 8723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:14.435031 8723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:14.435102 8723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:14.435155 8723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:14.444032 8723 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:37819
I20260810 07:20:14.444061 8918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:37819 every 8 connection(s)
I20260810 07:20:14.444950 8723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:14.450214 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 8723
I20260810 07:20:14.450364 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:14.450673 1527 external_mini_cluster.cc:1428] Setting key 4591a78fcf003669ee956d7496db9ca2
I20260810 07:20:14.451532 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.446891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:57931 (local address 127.1.125.254:39603)
0810 07:20:14.447038 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.447043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.448114 (+ 1071us) server_negotiation.cc:407] Connection header received
0810 07:20:14.449616 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.449620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.449667 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.449775 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.450989 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:14.450993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:14.450995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:14.451008 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:14.451033 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:14.451045 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:14.451139 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:14.451342 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 07:20:14.451403 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:20:14.452664 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.447301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39603 (local address 127.1.125.193:57931)
0810 07:20:14.447959 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:14.448005 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:14.449382 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:14.449949 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:14.449956 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:14.450041 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:14.450686 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:14.450687 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:14.450727 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:14.450829 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:14.451152 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:14.451161 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:14.451167 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:20:14.451375 (+ 208us) client_negotiation.cc:240] Negotiation successful
0810 07:20:14.451607 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":117,"threads_started":1}
I20260810 07:20:14.452809 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39603
--builtin_ntp_servers=127.1.125.212:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 07:20:14.453929 8919 heartbeater.cc:347] Connected to a master server at 127.1.125.254:39603
I20260810 07:20:14.454183 8919 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:14.454666 8919 heartbeater.cc:511] Master 127.1.125.254:39603 requested a full tablet report, sending...
I20260810 07:20:14.456359 8634 ts_manager.cc:194] Registered new tserver with Master: 41c1f1caba084786bff3cc4ecdbd03db (127.1.125.193:37819)
W20260810 07:20:14.573030 8922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:14.573331 8922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:14.573414 8922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:14.577895 8922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:14.577973 8922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:14.578061 8922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:14.583176 8922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:14.584273 8922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:14.585748 8922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:14.595835 8930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:14.595891 8931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:14.595860 8933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:14.596185 8922 server_base.cc:1034] running on GCE node
I20260810 07:20:14.596668 8922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:14.597365 8922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:14.598569 8922 hybrid_clock.cc:648] HybridClock initialized: now 1786346414598537 us; error 41 us; skew 500 ppm
I20260810 07:20:14.600466 8922 webserver.cc:460] Webserver started at http://127.1.125.194:33821/ using document root <none> and password file <none>
I20260810 07:20:14.601145 8922 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:14.601243 8922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:14.601495 8922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:14.603605 8922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "7b7a323b8a42411c83f29236b814bf49"
format_stamp: "Formatted at 2026-08-10 07:20:14 on dist-test-slave-44d2"
server_key: "225c896382870261d51507ead503d5ca"
server_key_iv: "2f34792c21dfc811e36f20df2ca5052c"
server_key_version: "encryptionkey@0"
I20260810 07:20:14.604185 8922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "7b7a323b8a42411c83f29236b814bf49"
format_stamp: "Formatted at 2026-08-10 07:20:14 on dist-test-slave-44d2"
server_key: "225c896382870261d51507ead503d5ca"
server_key_iv: "2f34792c21dfc811e36f20df2ca5052c"
server_key_version: "encryptionkey@0"
I20260810 07:20:14.607915 8922 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.003s
I20260810 07:20:14.610494 8941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:14.611864 8922 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:14.612035 8922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "7b7a323b8a42411c83f29236b814bf49"
format_stamp: "Formatted at 2026-08-10 07:20:14 on dist-test-slave-44d2"
server_key: "225c896382870261d51507ead503d5ca"
server_key_iv: "2f34792c21dfc811e36f20df2ca5052c"
server_key_version: "encryptionkey@0"
I20260810 07:20:14.612155 8922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:14.616770 8922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:14.617410 8922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:14.617605 8922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:14.618165 8922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:14.619084 8922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:14.619166 8922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:14.619234 8922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:14.619277 8922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:14.628499 8922 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:38365
I20260810 07:20:14.628592 9109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:38365 every 8 connection(s)
I20260810 07:20:14.629472 8922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:14.630596 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 8922
I20260810 07:20:14.630725 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:14.631124 1527 external_mini_cluster.cc:1428] Setting key 0876a349a8ad284bff3f2dc0ff29ffe0
I20260810 07:20:14.633291 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39603
--builtin_ntp_servers=127.1.125.212:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 07:20:14.636160 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.631550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:45019 (local address 127.1.125.254:39603)
0810 07:20:14.631708 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.631715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.633009 (+ 1294us) server_negotiation.cc:407] Connection header received
0810 07:20:14.634387 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.634392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.634441 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.634540 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.635687 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:14.635692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:14.635696 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:14.635713 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:14.635738 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:14.635750 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:14.635853 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:14.635954 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 07:20:14.636000 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:20:14.637037 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.632025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39603 (local address 127.1.125.194:45019)
0810 07:20:14.632836 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:14.632890 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:14.634179 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:14.634698 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:14.634704 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:14.634788 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:14.635420 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:14.635422 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:14.635462 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:14.635537 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:14.635844 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:14.635851 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:14.635856 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:20:14.635986 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 07:20:14.636170 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":183,"threads_started":1}
I20260810 07:20:14.638185 9110 heartbeater.cc:347] Connected to a master server at 127.1.125.254:39603
I20260810 07:20:14.638428 9110 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:14.638827 9110 heartbeater.cc:511] Master 127.1.125.254:39603 requested a full tablet report, sending...
I20260810 07:20:14.639829 8634 ts_manager.cc:194] Registered new tserver with Master: 7b7a323b8a42411c83f29236b814bf49 (127.1.125.194:38365)
W20260810 07:20:14.753352 9117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:14.753630 9117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:14.753698 9117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:14.757678 9117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:14.757759 9117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:14.757848 9117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:14.762566 9117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:14.763614 9117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:14.765007 9117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:20:14.774969 9117 server_base.cc:1034] running on GCE node
W20260810 07:20:14.774931 9128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:14.774931 9124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:14.774938 9125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:14.775664 9117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:14.776337 9117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:14.777532 9117 hybrid_clock.cc:648] HybridClock initialized: now 1786346414777516 us; error 32 us; skew 500 ppm
I20260810 07:20:14.779452 9117 webserver.cc:460] Webserver started at http://127.1.125.195:44753/ using document root <none> and password file <none>
I20260810 07:20:14.780050 9117 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:14.780145 9117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:14.780431 9117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:14.782565 9117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "af91d506c49147bdb8983114503b2e04"
format_stamp: "Formatted at 2026-08-10 07:20:14 on dist-test-slave-44d2"
server_key: "47a5d5c5a8e4300108541778f01ac9ef"
server_key_iv: "cf4cc92c2076ad259ecc633179daf41b"
server_key_version: "encryptionkey@0"
I20260810 07:20:14.783134 9117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "af91d506c49147bdb8983114503b2e04"
format_stamp: "Formatted at 2026-08-10 07:20:14 on dist-test-slave-44d2"
server_key: "47a5d5c5a8e4300108541778f01ac9ef"
server_key_iv: "cf4cc92c2076ad259ecc633179daf41b"
server_key_version: "encryptionkey@0"
I20260810 07:20:14.786842 9117 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 07:20:14.789302 9137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:14.790535 9117 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:20:14.790678 9117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "af91d506c49147bdb8983114503b2e04"
format_stamp: "Formatted at 2026-08-10 07:20:14 on dist-test-slave-44d2"
server_key: "47a5d5c5a8e4300108541778f01ac9ef"
server_key_iv: "cf4cc92c2076ad259ecc633179daf41b"
server_key_version: "encryptionkey@0"
I20260810 07:20:14.790789 9117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:14.795579 9117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:14.796183 9117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:14.796384 9117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:14.796955 9117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:14.797859 9117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:14.797931 9117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:14.798008 9117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:14.798050 9117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:14.811043 9117 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:43217
I20260810 07:20:14.812129 9117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:14.821733 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 9117
I20260810 07:20:14.821887 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:14.822248 1527 external_mini_cluster.cc:1428] Setting key 6d8fffef82ce1a2b227e3d52da30e3c5
I20260810 07:20:14.813154 9311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:43217 every 8 connection(s)
I20260810 07:20:14.823964 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.820701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:47009 (local address 127.1.125.254:39603)
0810 07:20:14.820824 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.820829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.820845 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:20:14.821410 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.821416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.821471 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.821518 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.823455 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:14.823458 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:14.823461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:14.823475 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:14.823502 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:14.823511 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:14.823600 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:14.823800 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 07:20:14.823847 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:20:14.825124 9315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.818790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39603 (local address 127.1.125.195:47009)
0810 07:20:14.819611 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:14.819668 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:14.821126 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:14.821946 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:14.821961 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:14.822104 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:14.823104 (+ 1000us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:14.823105 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:14.823168 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:14.823274 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:14.823616 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:14.823631 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:14.823641 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 07:20:14.823846 (+ 205us) client_negotiation.cc:240] Negotiation successful
0810 07:20:14.824121 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":143,"threads_started":1}
I20260810 07:20:14.826181 9312 heartbeater.cc:347] Connected to a master server at 127.1.125.254:39603
I20260810 07:20:14.826443 9312 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:14.826937 9312 heartbeater.cc:511] Master 127.1.125.254:39603 requested a full tablet report, sending...
I20260810 07:20:14.827970 8634 ts_manager.cc:194] Registered new tserver with Master: af91d506c49147bdb8983114503b2e04 (127.1.125.195:43217)
I20260810 07:20:14.836336 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:20:14.840030 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.838261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33968 (local address 127.1.125.254:39603)
0810 07:20:14.838414 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.838419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.838623 (+ 204us) server_negotiation.cc:407] Connection header received
0810 07:20:14.838783 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.838786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.838833 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.838914 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.839583 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:14.839587 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:14.839589 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:14.839601 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:14.839622 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:14.839638 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:14.839728 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:14.839838 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 07:20:14.839880 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:20:14.842782 8634 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:20:14.845942 8634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:14.857905 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.854429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:38365 (local address 127.0.0.1:40020)
0810 07:20:14.855216 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:14.855230 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:14.855286 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:14.856370 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:14.856373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:14.856389 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:14.856826 (+ 437us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:14.856827 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:14.856836 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:14.856850 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:14.857288 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:14.857301 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:14.857309 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:20:14.857404 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 07:20:14.857476 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":117,"threads_started":1}
I20260810 07:20:14.858009 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.854503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48856 (local address 127.1.125.195:43217)
0810 07:20:14.854944 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.854960 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.855394 (+ 434us) server_negotiation.cc:407] Connection header received
0810 07:20:14.855460 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.855465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.855684 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.855838 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.857072 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:14.857077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:14.857089 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:14.857101 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:14.857143 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:14.857177 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:14.857283 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:14.857728 (+ 445us) server_negotiation.cc:299] Negotiation successful
0810 07:20:14.857815 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":198,"threads_started":1}
I20260810 07:20:14.858309 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.855036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:37819 (local address 127.0.0.1:33404)
0810 07:20:14.855478 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:14.855490 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:14.855551 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:14.857165 (+ 1614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:14.857167 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:14.857181 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:14.857590 (+ 409us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:14.857592 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:14.857603 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:14.857631 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:14.858044 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:14.858047 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:14.858050 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:20:14.858139 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 07:20:14.858179 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":116,"threads_started":1}
I20260810 07:20:14.858529 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.854356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:43217 (local address 127.0.0.1:48856)
0810 07:20:14.855126 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:14.855178 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:14.855286 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:14.855838 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:14.855845 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:14.855922 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:14.856666 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:14.856668 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:14.856715 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:14.856937 (+ 222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:14.857573 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:14.857577 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:14.857580 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:20:14.858264 (+ 684us) client_negotiation.cc:240] Negotiation successful
0810 07:20:14.858306 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":59,"threads_started":1}
I20260810 07:20:14.858709 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.855638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40020 (local address 127.1.125.194:38365)
0810 07:20:14.855954 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.855966 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.855990 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:20:14.856043 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.856048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.856227 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.856370 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.857071 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:14.857077 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:14.857089 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:14.857100 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:14.857141 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:14.857168 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:14.858414 (+ 1246us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:14.858469 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 07:20:14.858539 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":136,"threads_started":1}
I20260810 07:20:14.858866 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.856042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33404 (local address 127.1.125.193:37819)
0810 07:20:14.856749 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.856761 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.856786 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:20:14.856841 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.856847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.857025 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.857183 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.857784 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:14.857794 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:14.857802 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:14.857830 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:14.857869 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:14.857894 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:14.858000 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:14.858620 (+ 620us) server_negotiation.cc:299] Negotiation successful
0810 07:20:14.858681 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":440,"threads_started":1}
I20260810 07:20:14.861443 9215 tablet_service.cc:1467] Processing CreateTablet for tablet 1a34b8e54ca043b793797de9af162bf0 (DEFAULT_TABLE table=test-table [id=c9567c982cdc4404886e4a6b11161c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:14.861466 8821 tablet_service.cc:1467] Processing CreateTablet for tablet 1a34b8e54ca043b793797de9af162bf0 (DEFAULT_TABLE table=test-table [id=c9567c982cdc4404886e4a6b11161c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:14.861467 9014 tablet_service.cc:1467] Processing CreateTablet for tablet 1a34b8e54ca043b793797de9af162bf0 (DEFAULT_TABLE table=test-table [id=c9567c982cdc4404886e4a6b11161c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:14.862505 9215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a34b8e54ca043b793797de9af162bf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:14.862529 8821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a34b8e54ca043b793797de9af162bf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:14.862529 9014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a34b8e54ca043b793797de9af162bf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:14.868651 9332 tablet_bootstrap.cc:492] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04: Bootstrap starting.
I20260810 07:20:14.870213 9333 tablet_bootstrap.cc:492] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49: Bootstrap starting.
I20260810 07:20:14.870409 9335 tablet_bootstrap.cc:492] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db: Bootstrap starting.
I20260810 07:20:14.871243 9332 tablet_bootstrap.cc:654] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:14.872000 9332 log.cc:824] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:14.872414 9335 tablet_bootstrap.cc:654] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:14.872414 9333 tablet_bootstrap.cc:654] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:14.873359 9335 log.cc:824] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:14.873359 9333 log.cc:824] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:14.874107 9332 tablet_bootstrap.cc:492] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04: No bootstrap required, opened a new log
I20260810 07:20:14.874305 9332 ts_tablet_manager.cc:1397] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:20:14.875080 9335 tablet_bootstrap.cc:492] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db: No bootstrap required, opened a new log
I20260810 07:20:14.875281 9335 ts_tablet_manager.cc:1397] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:20:14.876411 9333 tablet_bootstrap.cc:492] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49: No bootstrap required, opened a new log
I20260810 07:20:14.876617 9333 ts_tablet_manager.cc:1397] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 07:20:14.878062 9332 raft_consensus.cc:348] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af91d506c49147bdb8983114503b2e04" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43217 } } peers { permanent_uuid: "7b7a323b8a42411c83f29236b814bf49" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 38365 } } peers { permanent_uuid: "41c1f1caba084786bff3cc4ecdbd03db" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37819 } }
I20260810 07:20:14.878314 9332 raft_consensus.cc:374] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:14.878399 9332 raft_consensus.cc:729] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af91d506c49147bdb8983114503b2e04, State: Initialized, Role: FOLLOWER
I20260810 07:20:14.878873 9332 consensus_queue.cc:260] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af91d506c49147bdb8983114503b2e04" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43217 } } peers { permanent_uuid: "7b7a323b8a42411c83f29236b814bf49" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 38365 } } peers { permanent_uuid: "41c1f1caba084786bff3cc4ecdbd03db" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37819 } }
I20260810 07:20:14.879612 9312 heartbeater.cc:503] Master 127.1.125.254:39603 was elected leader, sending a full tablet report...
I20260810 07:20:14.879876 9332 ts_tablet_manager.cc:1428] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:20:14.880455 9335 raft_consensus.cc:348] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af91d506c49147bdb8983114503b2e04" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43217 } } peers { permanent_uuid: "7b7a323b8a42411c83f29236b814bf49" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 38365 } } peers { permanent_uuid: "41c1f1caba084786bff3cc4ecdbd03db" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37819 } }
I20260810 07:20:14.880762 9335 raft_consensus.cc:374] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:14.880889 9335 raft_consensus.cc:729] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c1f1caba084786bff3cc4ecdbd03db, State: Initialized, Role: FOLLOWER
I20260810 07:20:14.881183 9333 raft_consensus.cc:348] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af91d506c49147bdb8983114503b2e04" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43217 } } peers { permanent_uuid: "7b7a323b8a42411c83f29236b814bf49" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 38365 } } peers { permanent_uuid: "41c1f1caba084786bff3cc4ecdbd03db" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37819 } }
I20260810 07:20:14.881417 9333 raft_consensus.cc:374] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:14.881359 9335 consensus_queue.cc:260] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af91d506c49147bdb8983114503b2e04" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43217 } } peers { permanent_uuid: "7b7a323b8a42411c83f29236b814bf49" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 38365 } } peers { permanent_uuid: "41c1f1caba084786bff3cc4ecdbd03db" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37819 } }
I20260810 07:20:14.881477 9333 raft_consensus.cc:729] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b7a323b8a42411c83f29236b814bf49, State: Initialized, Role: FOLLOWER
I20260810 07:20:14.881922 9333 consensus_queue.cc:260] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af91d506c49147bdb8983114503b2e04" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43217 } } peers { permanent_uuid: "7b7a323b8a42411c83f29236b814bf49" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 38365 } } peers { permanent_uuid: "41c1f1caba084786bff3cc4ecdbd03db" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37819 } }
I20260810 07:20:14.882030 8919 heartbeater.cc:503] Master 127.1.125.254:39603 was elected leader, sending a full tablet report...
I20260810 07:20:14.882403 9335 ts_tablet_manager.cc:1428] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 07:20:14.882875 9333 ts_tablet_manager.cc:1428] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260810 07:20:14.883257 9110 heartbeater.cc:503] Master 127.1.125.254:39603 was elected leader, sending a full tablet report...
I20260810 07:20:14.888067 9344 raft_consensus.cc:482] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:14.888235 9344 raft_consensus.cc:504] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af91d506c49147bdb8983114503b2e04" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43217 } } peers { permanent_uuid: "7b7a323b8a42411c83f29236b814bf49" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 38365 } } peers { permanent_uuid: "41c1f1caba084786bff3cc4ecdbd03db" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37819 } }
I20260810 07:20:14.889318 9344 leader_election.cc:288] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af91d506c49147bdb8983114503b2e04 (127.1.125.195:43217), 41c1f1caba084786bff3cc4ecdbd03db (127.1.125.193:37819)
I20260810 07:20:14.891585 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.889687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:43217 (local address 127.1.125.194:44737)
0810 07:20:14.889967 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:14.889984 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:14.890064 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:14.890336 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:14.890339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:14.890354 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:14.890860 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:14.890919 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:14.890936 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:14.890971 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:14.891269 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:14.891273 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:14.891277 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:20:14.891379 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 07:20:14.891421 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260810 07:20:14.892112 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.889778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:52975 (local address 127.1.125.193:37819)
0810 07:20:14.889895 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.889900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.890279 (+ 379us) server_negotiation.cc:407] Connection header received
0810 07:20:14.890409 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.890413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.890448 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.890564 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.891580 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:14.891584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:14.891587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:14.891603 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:14.891630 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:14.891644 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:14.891716 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:14.891922 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 07:20:14.891992 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:20:14.892112 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.889778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:44737 (local address 127.1.125.195:43217)
0810 07:20:14.890010 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.890017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.890034 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:20:14.890186 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.890189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.890224 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.890310 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.891094 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:14.891098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:14.891104 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:14.891123 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:14.891147 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:14.891158 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:14.891251 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:14.891936 (+ 685us) server_negotiation.cc:299] Negotiation successful
0810 07:20:14.891979 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260810 07:20:14.892488 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.889687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:37819 (local address 127.1.125.194:52975)
0810 07:20:14.890165 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:14.890179 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:14.890304 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:14.890883 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:14.890886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:14.890899 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:14.891398 (+ 499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:14.891400 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:14.891413 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:14.891440 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:14.891808 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:14.891811 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:14.891814 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:20:14.892272 (+ 458us) client_negotiation.cc:240] Negotiation successful
0810 07:20:14.892312 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":36,"thread_start_us":210,"threads_started":1}
I20260810 07:20:14.892758 9241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a34b8e54ca043b793797de9af162bf0" candidate_uuid: "7b7a323b8a42411c83f29236b814bf49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af91d506c49147bdb8983114503b2e04" is_pre_election: true
I20260810 07:20:14.893011 9241 raft_consensus.cc:2454] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b7a323b8a42411c83f29236b814bf49 in term 0.
I20260810 07:20:14.893525 8946 leader_election.cc:302] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b7a323b8a42411c83f29236b814bf49, af91d506c49147bdb8983114503b2e04; no voters:
I20260810 07:20:14.893761 8852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a34b8e54ca043b793797de9af162bf0" candidate_uuid: "7b7a323b8a42411c83f29236b814bf49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c1f1caba084786bff3cc4ecdbd03db" is_pre_election: true
I20260810 07:20:14.893824 9344 raft_consensus.cc:2781] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:14.893935 9344 raft_consensus.cc:482] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:14.894004 9344 raft_consensus.cc:3037] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:14.894089 8852 raft_consensus.cc:2454] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b7a323b8a42411c83f29236b814bf49 in term 0.
I20260810 07:20:14.895377 9344 raft_consensus.cc:504] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af91d506c49147bdb8983114503b2e04" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43217 } } peers { permanent_uuid: "7b7a323b8a42411c83f29236b814bf49" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 38365 } } peers { permanent_uuid: "41c1f1caba084786bff3cc4ecdbd03db" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37819 } }
I20260810 07:20:14.895723 9344 leader_election.cc:288] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [CANDIDATE]: Term 1 election: Requested vote from peers af91d506c49147bdb8983114503b2e04 (127.1.125.195:43217), 41c1f1caba084786bff3cc4ecdbd03db (127.1.125.193:37819)
I20260810 07:20:14.896106 9241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a34b8e54ca043b793797de9af162bf0" candidate_uuid: "7b7a323b8a42411c83f29236b814bf49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af91d506c49147bdb8983114503b2e04"
I20260810 07:20:14.896209 9241 raft_consensus.cc:3037] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:14.896629 8852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a34b8e54ca043b793797de9af162bf0" candidate_uuid: "7b7a323b8a42411c83f29236b814bf49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c1f1caba084786bff3cc4ecdbd03db"
I20260810 07:20:14.896736 8852 raft_consensus.cc:3037] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:14.897460 9241 raft_consensus.cc:2454] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b7a323b8a42411c83f29236b814bf49 in term 1.
I20260810 07:20:14.897814 8946 leader_election.cc:302] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b7a323b8a42411c83f29236b814bf49, af91d506c49147bdb8983114503b2e04; no voters:
I20260810 07:20:14.898087 9344 raft_consensus.cc:2781] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:14.898082 8852 raft_consensus.cc:2454] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b7a323b8a42411c83f29236b814bf49 in term 1.
I20260810 07:20:14.898325 9344 raft_consensus.cc:686] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [term 1 LEADER]: Becoming Leader. State: Replica: 7b7a323b8a42411c83f29236b814bf49, State: Running, Role: LEADER
I20260810 07:20:14.898718 9344 consensus_queue.cc:237] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af91d506c49147bdb8983114503b2e04" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43217 } } peers { permanent_uuid: "7b7a323b8a42411c83f29236b814bf49" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 38365 } } peers { permanent_uuid: "41c1f1caba084786bff3cc4ecdbd03db" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37819 } }
I20260810 07:20:14.902036 8630 catalog_manager.cc:5495] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b7a323b8a42411c83f29236b814bf49 (127.1.125.194). New cstate: current_term: 1 leader_uuid: "7b7a323b8a42411c83f29236b814bf49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af91d506c49147bdb8983114503b2e04" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b7a323b8a42411c83f29236b814bf49" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 38365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41c1f1caba084786bff3cc4ecdbd03db" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37819 } health_report { overall_health: UNKNOWN } } }
I20260810 07:20:14.921129 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.919560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40034 (local address 127.1.125.194:38365)
0810 07:20:14.919699 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:14.919702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:14.919715 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:20:14.919820 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:14.919822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:14.919881 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:14.919949 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:14.920741 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:14.920743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:14.920745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:14.920754 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:14.920773 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:14.920781 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:14.920853 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:14.920916 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 07:20:14.920949 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:20:14.927194 8852 raft_consensus.cc:1261] T 1a34b8e54ca043b793797de9af162bf0 P 41c1f1caba084786bff3cc4ecdbd03db [term 1 FOLLOWER]: Refusing update from remote peer 7b7a323b8a42411c83f29236b814bf49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:14.927242 9241 raft_consensus.cc:1261] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04 [term 1 FOLLOWER]: Refusing update from remote peer 7b7a323b8a42411c83f29236b814bf49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:14.927917 9344 consensus_queue.cc:1035] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c1f1caba084786bff3cc4ecdbd03db" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:14.928119 9346 consensus_queue.cc:1035] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af91d506c49147bdb8983114503b2e04" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:14.935699 9350 mvcc.cc:205] Tried to move back new op lower bound from 7316874915533983744 to 7316874915427946496. Current Snapshot: MvccSnapshot[applied={T|T < 7316874915533983744}]
I20260810 07:20:14.943557 8630 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:14.943755 8630 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:14.946396 8630 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67cfaa0e65654a01a61e60676923cd1d: Sending DeleteTablet for 3 replicas of tablet 1a34b8e54ca043b793797de9af162bf0
I20260810 07:20:14.947230 9215 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a34b8e54ca043b793797de9af162bf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:14 UTC) from {username='slave'} at 127.0.0.1:48856
I20260810 07:20:14.947609 8821 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a34b8e54ca043b793797de9af162bf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:14 UTC) from {username='slave'} at 127.0.0.1:33404
I20260810 07:20:14.947927 9364 tablet_replica.cc:330] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04: stopping tablet replica
I20260810 07:20:14.948053 9014 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a34b8e54ca043b793797de9af162bf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:14 UTC) from {username='slave'} at 127.0.0.1:40020
I20260810 07:20:14.948104 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 8723
I20260810 07:20:14.948261 9364 raft_consensus.cc:2229] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:14.948670 9364 raft_consensus.cc:2258] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:14.951588 9366 tablet_replica.cc:330] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49: stopping tablet replica
I20260810 07:20:14.951967 9366 raft_consensus.cc:2229] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:14.952383 9366 raft_consensus.cc:2258] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:14.952620 9364 ts_tablet_manager.cc:1905] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:14.956604 9364 ts_tablet_manager.cc:1918] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:14.956776 9364 log.cc:1196] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/1a34b8e54ca043b793797de9af162bf0
I20260810 07:20:14.957190 9364 ts_tablet_manager.cc:1939] T 1a34b8e54ca043b793797de9af162bf0 P af91d506c49147bdb8983114503b2e04: Deleting consensus metadata
I20260810 07:20:14.958424 9366 ts_tablet_manager.cc:1905] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:14.960774 8614 catalog_manager.cc:4841] TS af91d506c49147bdb8983114503b2e04 (127.1.125.195:43217): tablet 1a34b8e54ca043b793797de9af162bf0 (table test-table [id=c9567c982cdc4404886e4a6b11161c6e]) successfully deleted
I20260810 07:20:14.961290 9366 ts_tablet_manager.cc:1918] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:14.961381 9366 log.cc:1196] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/1a34b8e54ca043b793797de9af162bf0
I20260810 07:20:14.961742 9366 ts_tablet_manager.cc:1939] T 1a34b8e54ca043b793797de9af162bf0 P 7b7a323b8a42411c83f29236b814bf49: Deleting consensus metadata
I20260810 07:20:14.963004 8611 catalog_manager.cc:4841] TS 7b7a323b8a42411c83f29236b814bf49 (127.1.125.194:38365): tablet 1a34b8e54ca043b793797de9af162bf0 (table test-table [id=c9567c982cdc4404886e4a6b11161c6e]) successfully deleted
W20260810 07:20:14.965253 8611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:37819 (error 108)
I20260810 07:20:14.965319 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 8922
I20260810 07:20:14.968632 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:14.968164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:37819 (local address 127.0.0.1:33406)
0810 07:20:14.968368 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:14.968523 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:37819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260810 07:20:14.968915 8611 catalog_manager.cc:4584] TS 41c1f1caba084786bff3cc4ecdbd03db (127.1.125.193:37819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1a34b8e54ca043b793797de9af162bf0: Network error: Client connection negotiation failed: client connection to 127.1.125.193:37819: connect: Connection refused (error 111)
I20260810 07:20:14.977051 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 9117
I20260810 07:20:14.989212 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 8585
2026-08-10T07:20:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (996 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 07:20:15.016244 1527 mini_oidc.cc:152] Starting JWKS server
I20260810 07:20:15.017969 1527 webserver.cc:460] Webserver started at https://127.0.0.1:35281/ using document root <none> and password file <none>
I20260810 07:20:15.018167 1527 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 07:20:15.018460 1527 webserver.cc:460] Webserver started at http://127.0.0.1:44887/ using document root <none> and password file <none>
2026-08-10T07:20:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:15Z Disabled control of system clock
I20260810 07:20:15.028942 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:36685
--webserver_interface=127.1.125.254
--webserver_port=0
--builtin_ntp_servers=127.1.125.212:46645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:36685
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35281/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/testchainca.cert with env {}
W20260810 07:20:15.144099 9380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:15.144385 9380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:15.144434 9380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:15.149096 9380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:15.149163 9380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:15.149180 9380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:15.149199 9380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:15.149216 9380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:20:15.149233 9380 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:20:15.149250 9380 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35281/jwks.json
I20260810 07:20:15.155200 9380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:46645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:36685
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:36685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35281/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:15.156261 9380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:15.157357 9380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:15.166553 9389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:15.166494 9386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:15.166618 9387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:15.166818 9380 server_base.cc:1034] running on GCE node
I20260810 07:20:15.167479 9380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:15.168671 9380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:15.169864 9380 hybrid_clock.cc:648] HybridClock initialized: now 1786346415169851 us; error 33 us; skew 500 ppm
I20260810 07:20:15.171873 9380 webserver.cc:460] Webserver started at http://127.1.125.254:45691/ using document root <none> and password file <none>
I20260810 07:20:15.172472 9380 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:15.172654 9380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:15.172958 9380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:15.175369 9380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "7210dbe304b14841b9be8d5916ed5549"
format_stamp: "Formatted at 2026-08-10 07:20:15 on dist-test-slave-44d2"
server_key: "2d1d4560c1e78efb1fce1e3eed4c016e"
server_key_iv: "8ae7500efa242a2f308e8a32030a93bb"
server_key_version: "encryptionkey@0"
I20260810 07:20:15.176107 9380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "7210dbe304b14841b9be8d5916ed5549"
format_stamp: "Formatted at 2026-08-10 07:20:15 on dist-test-slave-44d2"
server_key: "2d1d4560c1e78efb1fce1e3eed4c016e"
server_key_iv: "8ae7500efa242a2f308e8a32030a93bb"
server_key_version: "encryptionkey@0"
I20260810 07:20:15.180063 9380 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:20:15.182693 9397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:15.184023 9380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:15.184190 9380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "7210dbe304b14841b9be8d5916ed5549"
format_stamp: "Formatted at 2026-08-10 07:20:15 on dist-test-slave-44d2"
server_key: "2d1d4560c1e78efb1fce1e3eed4c016e"
server_key_iv: "8ae7500efa242a2f308e8a32030a93bb"
server_key_version: "encryptionkey@0"
I20260810 07:20:15.184304 9380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:15.217335 9380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:15.218192 9380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:15.218415 9380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:15.225793 9380 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:36685
I20260810 07:20:15.225816 9476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:36685 every 8 connection(s)
I20260810 07:20:15.226826 9380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:15.230142 9477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:15.235258 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 9380
I20260810 07:20:15.235404 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:15.235706 1527 external_mini_cluster.cc:1428] Setting key 07376f4aebcda4d135e43414c7662b44
I20260810 07:20:15.236172 9477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549: Bootstrap starting.
I20260810 07:20:15.239610 9477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:15.240397 9477 log.cc:824] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:15.242295 9477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549: No bootstrap required, opened a new log
I20260810 07:20:15.245193 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:15.237261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52114 (local address 127.1.125.254:36685)
0810 07:20:15.237905 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:15.237921 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:15.237958 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:20:15.238611 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:15.238628 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:15.239041 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:15.239338 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:15.239725 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.240821 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.241815 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.242300 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.242504 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.243355 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:15.243360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:15.243373 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:15.243755 (+ 382us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:15.243811 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:15.243831 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:15.243911 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:15.244105 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 07:20:15.244406 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":131,"threads_started":1}
I20260810 07:20:15.246281 9477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7210dbe304b14841b9be8d5916ed5549" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36685 } }
I20260810 07:20:15.246497 9477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:15.246574 9477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7210dbe304b14841b9be8d5916ed5549, State: Initialized, Role: FOLLOWER
I20260810 07:20:15.246990 9477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7210dbe304b14841b9be8d5916ed5549" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36685 } }
I20260810 07:20:15.247146 9477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:15.247218 9477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:15.247315 9477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:15.248201 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:20:15.248543 9477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7210dbe304b14841b9be8d5916ed5549" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36685 } }
I20260810 07:20:15.248906 9477 leader_election.cc:302] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7210dbe304b14841b9be8d5916ed5549; no voters:
I20260810 07:20:15.249298 9477 leader_election.cc:288] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:15.249372 9484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:15.249593 9484 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [term 1 LEADER]: Becoming Leader. State: Replica: 7210dbe304b14841b9be8d5916ed5549, State: Running, Role: LEADER
I20260810 07:20:15.249902 9484 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7210dbe304b14841b9be8d5916ed5549" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36685 } }
I20260810 07:20:15.250655 9477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:15.251703 9486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7210dbe304b14841b9be8d5916ed5549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7210dbe304b14841b9be8d5916ed5549" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36685 } } }
I20260810 07:20:15.251842 9486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:15.252254 9495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:15.252230 9485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7210dbe304b14841b9be8d5916ed5549. Latest consensus state: current_term: 1 leader_uuid: "7210dbe304b14841b9be8d5916ed5549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7210dbe304b14841b9be8d5916ed5549" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36685 } } }
I20260810 07:20:15.253002 9485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:15.255388 9495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:15.261484 9495 catalog_manager.cc:1347] Generated new cluster ID: 25148f55bc804f978a7fedd70c9aabab
I20260810 07:20:15.261564 9495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:15.283355 9495 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:15.284315 9495 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:15.304256 9495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7210dbe304b14841b9be8d5916ed5549: Generated new TSK 0
I20260810 07:20:15.305066 9495 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 07:20:15.335511 9517 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 07:20:15.335706 9517 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 07:20:15.336225 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:15.319347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52126 (local address 127.1.125.254:36685)
0810 07:20:15.319484 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:15.319489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:15.319697 (+ 208us) server_negotiation.cc:407] Connection header received
0810 07:20:15.319911 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:15.319913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:15.319959 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:15.320057 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:20:15.320372 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.320909 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.322171 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.322387 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.322462 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.322843 (+ 381us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:20:15.335857 (+ 13014us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 07:20:15.336113 (+ 256us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52126: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 07:20:15.336311 9481 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52126: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 07:20:15.337795 9516 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:15.319202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:36685 (local address 127.0.0.1:52126)
0810 07:20:15.319602 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:15.319619 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:15.319756 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:15.320103 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:15.320106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:15.320116 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:20:15.320256 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:15.320263 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.321045 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.321049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:15.322057 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:15.322068 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.322507 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.322513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:15.322610 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.322662 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:20:15.336095 (+ 13433us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 07:20:15.336350 (+ 255us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.125.254:36685: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":113,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.125.254:36685: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 07:20:15.338752 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 9380
2026-08-10T07:20:15Z chronyd exiting
I20260810 07:20:15.386497 1527 test_util.cc:175] -----------------------------------------------
I20260810 07:20:15.386603 1527 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346386457763-1527-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (372 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 07:20:15.387790 1527 mini_oidc.cc:152] Starting JWKS server
I20260810 07:20:15.388796 1527 webserver.cc:460] Webserver started at https://127.0.0.1:34605/ using document root <none> and password file <none>
I20260810 07:20:15.388923 1527 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 07:20:15.389214 1527 webserver.cc:460] Webserver started at http://127.0.0.1:35783/ using document root <none> and password file <none>
2026-08-10T07:20:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:15Z Disabled control of system clock
I20260810 07:20:15.400261 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:43961
--webserver_interface=127.1.125.254
--webserver_port=0
--builtin_ntp_servers=127.1.125.212:42301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:43961
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34605/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/testchainca.cert with env {}
W20260810 07:20:15.524796 9524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:15.525256 9524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:15.525302 9524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:15.529565 9524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:15.529629 9524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:15.529645 9524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:15.529662 9524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:15.529727 9524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:20:15.529750 9524 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:20:15.529786 9524 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34605/jwks.json
I20260810 07:20:15.535413 9524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:43961
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:43961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34605/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:15.536684 9524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:15.537765 9524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:15.546597 9530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:15.546806 9524 server_base.cc:1034] running on GCE node
W20260810 07:20:15.546645 9533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:15.546604 9531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:15.547477 9524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:15.548777 9524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:15.549917 9524 hybrid_clock.cc:648] HybridClock initialized: now 1786346415549890 us; error 70 us; skew 500 ppm
I20260810 07:20:15.552526 9524 webserver.cc:460] Webserver started at http://127.1.125.254:44415/ using document root <none> and password file <none>
I20260810 07:20:15.553181 9524 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:15.553282 9524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:15.553510 9524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:15.555637 9524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "6162579bc3c446caaed7a9d08960a4c3"
format_stamp: "Formatted at 2026-08-10 07:20:15 on dist-test-slave-44d2"
server_key: "314fca74dac4cca62b1d7f10f1f58ecb"
server_key_iv: "31a9fa01d0dbc5285157892daaedf732"
server_key_version: "encryptionkey@0"
I20260810 07:20:15.556298 9524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "6162579bc3c446caaed7a9d08960a4c3"
format_stamp: "Formatted at 2026-08-10 07:20:15 on dist-test-slave-44d2"
server_key: "314fca74dac4cca62b1d7f10f1f58ecb"
server_key_iv: "31a9fa01d0dbc5285157892daaedf732"
server_key_version: "encryptionkey@0"
I20260810 07:20:15.560010 9524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:20:15.562664 9541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:15.563853 9524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:20:15.564002 9524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "6162579bc3c446caaed7a9d08960a4c3"
format_stamp: "Formatted at 2026-08-10 07:20:15 on dist-test-slave-44d2"
server_key: "314fca74dac4cca62b1d7f10f1f58ecb"
server_key_iv: "31a9fa01d0dbc5285157892daaedf732"
server_key_version: "encryptionkey@0"
I20260810 07:20:15.564127 9524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:15.580005 9524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:15.580988 9524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:15.581231 9524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:15.588266 9524 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:43961
I20260810 07:20:15.588284 9626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:43961 every 8 connection(s)
I20260810 07:20:15.589272 9524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:15.592731 9627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:15.597074 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 9524
I20260810 07:20:15.597185 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:15.597479 1527 external_mini_cluster.cc:1428] Setting key 1b65e05ef0eee68c0137553adbdfa4e1
I20260810 07:20:15.599143 9627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3: Bootstrap starting.
I20260810 07:20:15.601766 9627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:15.602577 9627 log.cc:824] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:15.604679 9627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3: No bootstrap required, opened a new log
I20260810 07:20:15.606874 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:15.598996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37896 (local address 127.1.125.254:43961)
0810 07:20:15.599638 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:15.599653 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:15.599691 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:20:15.600365 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:15.600382 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:15.600873 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:15.601157 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:15.601610 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.602635 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.603554 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.604035 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.604220 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.605179 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:15.605184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:15.605197 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:15.605588 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:15.605644 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:15.605666 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:15.605783 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:15.605902 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 07:20:15.606167 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":151,"threads_started":1}
I20260810 07:20:15.608979 9627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6162579bc3c446caaed7a9d08960a4c3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43961 } }
I20260810 07:20:15.609297 9627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:15.609397 9627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6162579bc3c446caaed7a9d08960a4c3, State: Initialized, Role: FOLLOWER
I20260810 07:20:15.609915 9627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6162579bc3c446caaed7a9d08960a4c3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43961 } }
I20260810 07:20:15.610103 9627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:15.610189 9627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:15.610198 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:20:15.610306 9627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:15.611482 9627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6162579bc3c446caaed7a9d08960a4c3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43961 } }
I20260810 07:20:15.611866 9627 leader_election.cc:302] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6162579bc3c446caaed7a9d08960a4c3; no voters:
I20260810 07:20:15.612264 9627 leader_election.cc:288] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:15.612362 9634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:15.612679 9634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [term 1 LEADER]: Becoming Leader. State: Replica: 6162579bc3c446caaed7a9d08960a4c3, State: Running, Role: LEADER
I20260810 07:20:15.613062 9634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6162579bc3c446caaed7a9d08960a4c3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43961 } }
I20260810 07:20:15.613648 9627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:15.614931 9636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6162579bc3c446caaed7a9d08960a4c3. Latest consensus state: current_term: 1 leader_uuid: "6162579bc3c446caaed7a9d08960a4c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6162579bc3c446caaed7a9d08960a4c3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43961 } } }
I20260810 07:20:15.614931 9635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6162579bc3c446caaed7a9d08960a4c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6162579bc3c446caaed7a9d08960a4c3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43961 } } }
I20260810 07:20:15.615180 9635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:15.615180 9636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:15.615640 9645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:15.618950 9645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:15.624775 9645 catalog_manager.cc:1347] Generated new cluster ID: b38bdacd965244ca83a8202270612f4a
I20260810 07:20:15.624900 9645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:15.636703 9645 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:15.637959 9645 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:15.643615 9645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6162579bc3c446caaed7a9d08960a4c3: Generated new TSK 0
I20260810 07:20:15.644264 9645 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 07:20:15.668289 9666 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 07:20:15.668378 9666 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 07:20:15.669013 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:15.653133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37898 (local address 127.1.125.254:43961)
0810 07:20:15.653276 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:15.653280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:15.653512 (+ 232us) server_negotiation.cc:407] Connection header received
0810 07:20:15.653681 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:15.653683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:15.653729 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:15.653817 (+ 88us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:20:15.654118 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.654625 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.655875 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.656093 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.656169 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.656623 (+ 454us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:20:15.668693 (+ 12070us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 07:20:15.668896 (+ 203us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37898: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 07:20:15.669107 9632 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37898: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 07:20:15.669301 9665 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:15.653066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:43961 (local address 127.0.0.1:37898)
0810 07:20:15.653410 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:15.653428 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:15.653556 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:15.653846 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:15.653849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:15.653858 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:20:15.653998 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:15.654006 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.654758 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.654762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:15.655750 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:15.655762 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.656213 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.656218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:15.656356 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.656390 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:20:15.668910 (+ 12520us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 07:20:15.669091 (+ 181us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.125.254:43961: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":123,"threads_started":1}
I20260810 07:20:15.669982 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 9524
2026-08-10T07:20:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (328 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 07:20:15.716167 1527 mini_oidc.cc:152] Starting JWKS server
I20260810 07:20:15.717146 1527 webserver.cc:460] Webserver started at https://127.0.0.1:45989/ using document root <none> and password file <none>
I20260810 07:20:15.717289 1527 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 07:20:15.717568 1527 webserver.cc:460] Webserver started at http://127.0.0.1:46653/ using document root <none> and password file <none>
2026-08-10T07:20:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:15Z Disabled control of system clock
I20260810 07:20:15.728385 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:32797
--webserver_interface=127.1.125.254
--webserver_port=0
--builtin_ntp_servers=127.1.125.212:43139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:32797
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45989/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 07:20:15.841876 9673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:15.842157 9673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:15.842195 9673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:15.846230 9673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:15.846292 9673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:15.846307 9673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:15.846323 9673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:15.846339 9673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:20:15.846354 9673 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:20:15.846367 9673 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45989/jwks.json
I20260810 07:20:15.851238 9673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:32797
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:32797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45989/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:15.852288 9673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:15.853720 9673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:15.863044 9681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:15.863150 9673 server_base.cc:1034] running on GCE node
W20260810 07:20:15.863072 9684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:15.863075 9680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:15.863852 9673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:15.865172 9673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:15.866405 9673 hybrid_clock.cc:648] HybridClock initialized: now 1786346415866385 us; error 26 us; skew 500 ppm
I20260810 07:20:15.868542 9673 webserver.cc:460] Webserver started at http://127.1.125.254:45643/ using document root <none> and password file <none>
I20260810 07:20:15.869197 9673 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:15.869302 9673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:15.869562 9673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:15.871825 9673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "267c839f1def49428593ff2b5ba79cb3"
format_stamp: "Formatted at 2026-08-10 07:20:15 on dist-test-slave-44d2"
server_key: "357f45e85bfb70efdcd1a59da322170e"
server_key_iv: "da9f61f5ca98dca33b801a54bb579b37"
server_key_version: "encryptionkey@0"
I20260810 07:20:15.872578 9673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "267c839f1def49428593ff2b5ba79cb3"
format_stamp: "Formatted at 2026-08-10 07:20:15 on dist-test-slave-44d2"
server_key: "357f45e85bfb70efdcd1a59da322170e"
server_key_iv: "da9f61f5ca98dca33b801a54bb579b37"
server_key_version: "encryptionkey@0"
I20260810 07:20:15.876669 9673 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:20:15.879372 9692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:15.881014 9673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:15.881178 9673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "267c839f1def49428593ff2b5ba79cb3"
format_stamp: "Formatted at 2026-08-10 07:20:15 on dist-test-slave-44d2"
server_key: "357f45e85bfb70efdcd1a59da322170e"
server_key_iv: "da9f61f5ca98dca33b801a54bb579b37"
server_key_version: "encryptionkey@0"
I20260810 07:20:15.881306 9673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:15.897845 9673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:15.898594 9673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:15.898824 9673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:15.906939 9673 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:32797
I20260810 07:20:15.906945 9763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:32797 every 8 connection(s)
I20260810 07:20:15.908016 9673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:15.911298 9766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:15.914727 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 9673
I20260810 07:20:15.914942 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:15.915257 1527 external_mini_cluster.cc:1428] Setting key 1f556fc271d15ac5f6fb8fb789083d24
I20260810 07:20:15.918045 9766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3: Bootstrap starting.
I20260810 07:20:15.920414 9766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:15.921283 9766 log.cc:824] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:15.923504 9766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3: No bootstrap required, opened a new log
I20260810 07:20:15.925526 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:15.916838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56096 (local address 127.1.125.254:32797)
0810 07:20:15.917628 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:15.917646 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:15.917685 (+ 39us) server_negotiation.cc:407] Connection header received
0810 07:20:15.918522 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:15.918542 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:15.918951 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:15.919289 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:15.919742 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.920900 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.921873 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.922395 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.922575 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.923532 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:15.923538 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:15.923551 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:15.924024 (+ 473us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:15.924088 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:15.924110 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:15.924230 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:15.924433 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 07:20:15.924742 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":149,"threads_started":1}
I20260810 07:20:15.927681 9766 raft_consensus.cc:348] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267c839f1def49428593ff2b5ba79cb3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 32797 } }
I20260810 07:20:15.927958 9766 raft_consensus.cc:374] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:15.928045 9766 raft_consensus.cc:729] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267c839f1def49428593ff2b5ba79cb3, State: Initialized, Role: FOLLOWER
I20260810 07:20:15.928565 9766 consensus_queue.cc:260] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267c839f1def49428593ff2b5ba79cb3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 32797 } }
I20260810 07:20:15.928748 9766 raft_consensus.cc:388] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:15.928839 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:20:15.928854 9766 raft_consensus.cc:482] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:15.928962 9766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:15.929965 9766 raft_consensus.cc:504] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267c839f1def49428593ff2b5ba79cb3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 32797 } }
I20260810 07:20:15.930356 9766 leader_election.cc:302] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 267c839f1def49428593ff2b5ba79cb3; no voters:
I20260810 07:20:15.930773 9766 leader_election.cc:288] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:15.930883 9776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:15.931164 9776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [term 1 LEADER]: Becoming Leader. State: Replica: 267c839f1def49428593ff2b5ba79cb3, State: Running, Role: LEADER
I20260810 07:20:15.931591 9776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267c839f1def49428593ff2b5ba79cb3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 32797 } }
I20260810 07:20:15.932129 9766 sys_catalog.cc:564] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:15.933616 9780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 267c839f1def49428593ff2b5ba79cb3. Latest consensus state: current_term: 1 leader_uuid: "267c839f1def49428593ff2b5ba79cb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267c839f1def49428593ff2b5ba79cb3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 32797 } } }
I20260810 07:20:15.933616 9778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "267c839f1def49428593ff2b5ba79cb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267c839f1def49428593ff2b5ba79cb3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 32797 } } }
I20260810 07:20:15.933784 9780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:15.933784 9778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:15.934729 9791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:15.938230 9791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:15.944164 9791 catalog_manager.cc:1347] Generated new cluster ID: eb173508582b4c9ea99c138de7df94ad
I20260810 07:20:15.944260 9791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:15.968658 9791 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:15.969686 9791 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:15.977041 9791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 267c839f1def49428593ff2b5ba79cb3: Generated new TSK 0
I20260810 07:20:15.977922 9791 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:20:16.001061 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:15.995235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56104 (local address 127.1.125.254:32797)
0810 07:20:15.995400 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:15.995406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:15.995602 (+ 196us) server_negotiation.cc:407] Connection header received
0810 07:20:15.995773 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:15.995776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:15.995829 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:15.995987 (+ 158us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:20:15.996289 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.996982 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.998262 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.998501 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.998603 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.998951 (+ 348us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:20:16.000628 (+ 1677us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45989/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 07:20:16.000889 (+ 261us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56104: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45989/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":64}
W20260810 07:20:16.001147 9773 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56104: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45989/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 07:20:16.002877 9808 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:15.995149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:32797 (local address 127.0.0.1:56104)
0810 07:20:15.995500 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:15.995516 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:15.995635 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:15.996009 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:15.996012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:15.996022 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:20:15.996157 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:15.996164 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.997112 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.997116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:15.998124 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:15.998135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:15.998641 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:15.998645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:15.998740 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:15.998769 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:20:16.000829 (+ 2060us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45989/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 07:20:16.002692 (+ 1863us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.125.254:32797: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45989/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":1632,"thread_start_us":133,"threads_started":1}
I20260810 07:20:16.003557 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 9673
E20260810 07:20:16.018314 9809 webserver.cc:533] Webserver: sslize failed:
E20260810 07:20:16.018445 9809 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T07:20:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (336 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T07:20:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:16Z Disabled control of system clock
I20260810 07:20:16.063871 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:36905
--webserver_interface=127.1.125.254
--webserver_port=0
--builtin_ntp_servers=127.1.125.212:34101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:36905
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/testchainca.cert with env {}
W20260810 07:20:16.179760 9815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:16.180030 9815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:16.180071 9815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:16.183983 9815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:16.184041 9815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:16.184057 9815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:16.184074 9815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:16.184089 9815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:20:16.184104 9815 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:20:16.184118 9815 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 07:20:16.189347 9815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:34101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:36905
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:36905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:16.190373 9815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:16.191390 9815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:16.200201 9822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:16.200416 9815 server_base.cc:1034] running on GCE node
W20260810 07:20:16.200246 9824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:16.200343 9820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:16.201198 9815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:16.202370 9815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:16.203583 9815 hybrid_clock.cc:648] HybridClock initialized: now 1786346416203563 us; error 27 us; skew 500 ppm
I20260810 07:20:16.205511 9815 webserver.cc:460] Webserver started at http://127.1.125.254:41759/ using document root <none> and password file <none>
I20260810 07:20:16.206138 9815 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:16.206241 9815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:16.206485 9815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:16.208669 9815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "b3fd53ce6ff74dc89ce045b712c9288d"
format_stamp: "Formatted at 2026-08-10 07:20:16 on dist-test-slave-44d2"
server_key: "ca7112dc506d2170994d1efead322be4"
server_key_iv: "958fd453f171976602262592c4cb813d"
server_key_version: "encryptionkey@0"
I20260810 07:20:16.209276 9815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "b3fd53ce6ff74dc89ce045b712c9288d"
format_stamp: "Formatted at 2026-08-10 07:20:16 on dist-test-slave-44d2"
server_key: "ca7112dc506d2170994d1efead322be4"
server_key_iv: "958fd453f171976602262592c4cb813d"
server_key_version: "encryptionkey@0"
I20260810 07:20:16.213024 9815 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260810 07:20:16.215540 9838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:16.216869 9815 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:16.217017 9815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "b3fd53ce6ff74dc89ce045b712c9288d"
format_stamp: "Formatted at 2026-08-10 07:20:16 on dist-test-slave-44d2"
server_key: "ca7112dc506d2170994d1efead322be4"
server_key_iv: "958fd453f171976602262592c4cb813d"
server_key_version: "encryptionkey@0"
I20260810 07:20:16.217137 9815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:16.238423 9815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:16.239185 9815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:16.239409 9815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:16.246990 9815 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:36905
I20260810 07:20:16.247115 9908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:36905 every 8 connection(s)
I20260810 07:20:16.248219 9815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:16.250119 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 9815
I20260810 07:20:16.250257 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:16.250598 1527 external_mini_cluster.cc:1428] Setting key e05b38f67a470b5ab36734d4871801ce
I20260810 07:20:16.252014 9909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:16.257987 9909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d: Bootstrap starting.
I20260810 07:20:16.260373 9909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:16.260390 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:16.252114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43954 (local address 127.1.125.254:36905)
0810 07:20:16.252877 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:16.252895 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:16.252932 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:20:16.253596 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:16.253614 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:16.254076 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:16.254425 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:16.254898 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.255883 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.256874 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.257420 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.257616 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:16.258609 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:16.258615 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:16.258627 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:16.259014 (+ 387us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:16.259073 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:16.259096 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:16.259206 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:16.259352 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 07:20:16.259624 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":146,"threads_started":1}
I20260810 07:20:16.261174 9909 log.cc:824] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:16.263510 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:20:16.263536 9909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d: No bootstrap required, opened a new log
I20260810 07:20:16.267318 9909 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3fd53ce6ff74dc89ce045b712c9288d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36905 } }
I20260810 07:20:16.267570 9909 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:16.267632 9909 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3fd53ce6ff74dc89ce045b712c9288d, State: Initialized, Role: FOLLOWER
I20260810 07:20:16.268036 9909 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3fd53ce6ff74dc89ce045b712c9288d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36905 } }
I20260810 07:20:16.268189 9909 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:16.268278 9909 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:16.268378 9909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:16.269306 9909 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3fd53ce6ff74dc89ce045b712c9288d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36905 } }
I20260810 07:20:16.269678 9909 leader_election.cc:302] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3fd53ce6ff74dc89ce045b712c9288d; no voters:
I20260810 07:20:16.270025 9909 leader_election.cc:288] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:16.270138 9917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:16.270387 9917 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [term 1 LEADER]: Becoming Leader. State: Replica: b3fd53ce6ff74dc89ce045b712c9288d, State: Running, Role: LEADER
I20260810 07:20:16.270972 9917 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3fd53ce6ff74dc89ce045b712c9288d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36905 } }
I20260810 07:20:16.271415 9909 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:16.272781 9921 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3fd53ce6ff74dc89ce045b712c9288d. Latest consensus state: current_term: 1 leader_uuid: "b3fd53ce6ff74dc89ce045b712c9288d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3fd53ce6ff74dc89ce045b712c9288d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36905 } } }
I20260810 07:20:16.272928 9921 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:16.273102 9920 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3fd53ce6ff74dc89ce045b712c9288d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3fd53ce6ff74dc89ce045b712c9288d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36905 } } }
I20260810 07:20:16.273255 9920 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:16.273393 9930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:16.276087 9930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:16.281692 9930 catalog_manager.cc:1347] Generated new cluster ID: 3ce32436b8a64a13911fae96b2f6aa6b
I20260810 07:20:16.281762 9930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:16.289776 9930 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:16.290741 9930 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:16.296294 9930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3fd53ce6ff74dc89ce045b712c9288d: Generated new TSK 0
I20260810 07:20:16.297200 9930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:20:16.443665 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:16.315187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43964 (local address 127.1.125.254:36905)
0810 07:20:16.315345 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:16.315350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:16.315506 (+ 156us) server_negotiation.cc:407] Connection header received
0810 07:20:16.315669 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:16.315673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:16.315733 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:16.315857 (+ 124us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:20:16.316088 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.316968 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.317903 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.318250 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.318355 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:16.318708 (+ 353us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:20:16.443227 (+124519us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 07:20:16.443522 (+ 295us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43964: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 07:20:16.443745 9914 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43964: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 07:20:16.443921 9951 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:16.315098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:36905 (local address 127.0.0.1:43964)
0810 07:20:16.315411 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:16.315425 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:16.315551 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:16.315862 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:16.315865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:16.315871 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:20:16.315973 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:16.315977 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.317112 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.317115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:16.317768 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:16.317777 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:16.318400 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:16.318404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:16.318483 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:16.318509 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:20:16.443475 (+124966us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 07:20:16.443681 (+ 206us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.125.254:36905: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":115,"threads_started":1}
I20260810 07:20:16.444870 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 9815
2026-08-10T07:20:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (431 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (130 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (131 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:16 dist-test-slave-44d2 krb5kdc[9969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:16 dist-test-slave-44d2 krb5kdc[9969](info): set up 2 sockets
Aug 10 07:20:16 dist-test-slave-44d2 krb5kdc[9969](info): commencing operation
krb5kdc: starting...
W20260810 07:20:18.068668 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:20:18 dist-test-slave-44d2 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:20:18.195904 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38731
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:43801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:38731
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:18.316007 9988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:18.316345 9988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:18.316444 9988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:18.320616 9988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:18.320709 9988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:18.320751 9988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:18.320811 9988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:18.320850 9988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:18.326192 9988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:38731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:18.327277 9988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:18.328354 9988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:18.337218 9995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:18.337250 9993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:18.337250 9999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:18.337544 9988 server_base.cc:1034] running on GCE node
I20260810 07:20:18.338119 9988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:18.339262 9988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:18.340456 9988 hybrid_clock.cc:648] HybridClock initialized: now 1786346418340427 us; error 39 us; skew 500 ppm
Aug 10 07:20:18 dist-test-slave-44d2 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:18.351651 9988 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:20:18.352938 9988 webserver.cc:460] Webserver started at http://127.1.125.254:43311/ using document root <none> and password file <none>
I20260810 07:20:18.353615 9988 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:18.353686 9988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:18.353967 9988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:18.356442 9988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "f73951393e644d3089b82032b40b463c"
format_stamp: "Formatted at 2026-08-10 07:20:18 on dist-test-slave-44d2"
server_key: "e1fb018730f394ba5292552ec5441d62"
server_key_iv: "0d0900383dfda8373b19c155aaad908f"
server_key_version: "encryptionkey@0"
I20260810 07:20:18.357152 9988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "f73951393e644d3089b82032b40b463c"
format_stamp: "Formatted at 2026-08-10 07:20:18 on dist-test-slave-44d2"
server_key: "e1fb018730f394ba5292552ec5441d62"
server_key_iv: "0d0900383dfda8373b19c155aaad908f"
server_key_version: "encryptionkey@0"
I20260810 07:20:18.361356 9988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 07:20:18.364044 10007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:18.365638 9988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:18.365815 9988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "f73951393e644d3089b82032b40b463c"
format_stamp: "Formatted at 2026-08-10 07:20:18 on dist-test-slave-44d2"
server_key: "e1fb018730f394ba5292552ec5441d62"
server_key_iv: "0d0900383dfda8373b19c155aaad908f"
server_key_version: "encryptionkey@0"
I20260810 07:20:18.365980 9988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:18.391113 9988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:18.394555 9988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:18.394798 9988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:18.403123 9988 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:38731
I20260810 07:20:18.403153 10083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:38731 every 8 connection(s)
I20260810 07:20:18.404227 9988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:18.407150 10084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:18.412978 10084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c: Bootstrap starting.
I20260810 07:20:18.413203 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 9988
I20260810 07:20:18.413343 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:18.413650 1527 external_mini_cluster.cc:1428] Setting key cbd12bad1ad9be9078b87f04ef6e3748
I20260810 07:20:18.415808 10084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:18.416636 10084 log.cc:824] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:18.418558 10084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c: No bootstrap required, opened a new log
I20260810 07:20:18.422219 10084 raft_consensus.cc:348] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73951393e644d3089b82032b40b463c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38731 } }
I20260810 07:20:18.422508 10084 raft_consensus.cc:374] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:18.422570 10084 raft_consensus.cc:729] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f73951393e644d3089b82032b40b463c, State: Initialized, Role: FOLLOWER
Aug 10 07:20:18 dist-test-slave-44d2 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:18.423097 10084 consensus_queue.cc:260] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73951393e644d3089b82032b40b463c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38731 } }
I20260810 07:20:18.423276 10084 raft_consensus.cc:388] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:18.423383 10084 raft_consensus.cc:482] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:18.423514 10084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:18.424412 10084 raft_consensus.cc:504] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73951393e644d3089b82032b40b463c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38731 } }
I20260810 07:20:18.424829 10084 leader_election.cc:302] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f73951393e644d3089b82032b40b463c; no voters:
I20260810 07:20:18.425185 10084 leader_election.cc:288] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:18.425325 10092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:18.425588 10092 raft_consensus.cc:686] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [term 1 LEADER]: Becoming Leader. State: Replica: f73951393e644d3089b82032b40b463c, State: Running, Role: LEADER
I20260810 07:20:18.425894 10092 consensus_queue.cc:237] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73951393e644d3089b82032b40b463c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38731 } }
I20260810 07:20:18.426640 10084 sys_catalog.cc:564] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:18.427513 10094 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f73951393e644d3089b82032b40b463c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73951393e644d3089b82032b40b463c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38731 } } }
I20260810 07:20:18.427634 10094 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:18.428020 10095 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f73951393e644d3089b82032b40b463c. Latest consensus state: current_term: 1 leader_uuid: "f73951393e644d3089b82032b40b463c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73951393e644d3089b82032b40b463c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38731 } } }
I20260810 07:20:18.428175 10095 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:18.429234 10110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:18.430367 10089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:18.415069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43708 (local address 127.1.125.254:38731)
0810 07:20:18.415540 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:18.415555 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:18.415592 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:20:18.416293 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:18.416312 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:18.416730 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:18.417082 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:18.417960 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.419002 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.419860 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.420334 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.420537 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:18.423423 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:18.423429 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:18.423447 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:18.423502 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:18.425683 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:18.426226 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:18.426233 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:18.426239 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:18.426383 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:18.426721 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:18.426728 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:18.426729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:18.427274 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:18.427490 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:18.428452 (+ 962us) server_negotiation.cc:299] Negotiation successful
0810 07:20:18.428815 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":134,"threads_started":1}
I20260810 07:20:18.432267 10110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:18.437309 10110 catalog_manager.cc:1347] Generated new cluster ID: 09cddbebca324242baec3ac72e62e0a7
I20260810 07:20:18.437374 10110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:18.448037 10110 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:18.449236 10110 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:18.457374 10110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f73951393e644d3089b82032b40b463c: Generated new TSK 0
I20260810 07:20:18.458107 10110 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:20:18.495738 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38731
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:43801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:18.614526 10132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:18.614775 10132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:18.614818 10132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:18.618816 10132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:18.618872 10132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:18.618927 10132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:18.624119 10132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:18.625144 10132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:18.626174 10132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:18.635639 10138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:18.635633 10139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:18.635681 10142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:18.636003 10132 server_base.cc:1034] running on GCE node
I20260810 07:20:18.636377 10132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:18.637065 10132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:18.638227 10132 hybrid_clock.cc:648] HybridClock initialized: now 1786346418638211 us; error 25 us; skew 500 ppm
Aug 10 07:20:18 dist-test-slave-44d2 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:18.641006 10132 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:20:18.641916 10132 webserver.cc:460] Webserver started at http://127.1.125.193:43655/ using document root <none> and password file <none>
I20260810 07:20:18.642459 10132 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:18.642519 10132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:18.642798 10132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:18.644985 10132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "40bc4c4f76464bf1a8be37a7eaa0a564"
format_stamp: "Formatted at 2026-08-10 07:20:18 on dist-test-slave-44d2"
server_key: "dddfd47334df06444e2071cd91f1a8b2"
server_key_iv: "2c210d81f9c8b64bc1b19b9223e4fbe9"
server_key_version: "encryptionkey@0"
I20260810 07:20:18.645615 10132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "40bc4c4f76464bf1a8be37a7eaa0a564"
format_stamp: "Formatted at 2026-08-10 07:20:18 on dist-test-slave-44d2"
server_key: "dddfd47334df06444e2071cd91f1a8b2"
server_key_iv: "2c210d81f9c8b64bc1b19b9223e4fbe9"
server_key_version: "encryptionkey@0"
I20260810 07:20:18.650058 10132 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:20:18.653205 10151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:18.654891 10132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:18.655032 10132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "40bc4c4f76464bf1a8be37a7eaa0a564"
format_stamp: "Formatted at 2026-08-10 07:20:18 on dist-test-slave-44d2"
server_key: "dddfd47334df06444e2071cd91f1a8b2"
server_key_iv: "2c210d81f9c8b64bc1b19b9223e4fbe9"
server_key_version: "encryptionkey@0"
I20260810 07:20:18.655161 10132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:18.670989 10132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:18.674219 10132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:18.674441 10132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:18.675081 10132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:18.675971 10132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:18.676044 10132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:18.676122 10132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:18.676172 10132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:18.686501 10132 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:40819
I20260810 07:20:18.686522 10316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:40819 every 8 connection(s)
I20260810 07:20:18.687445 10132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:18.692813 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 10132
I20260810 07:20:18.692950 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:18.693328 1527 external_mini_cluster.cc:1428] Setting key f7f5fe591ef52c6e640a5be7bbdb8298
Aug 10 07:20:18 dist-test-slave-44d2 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:18.701961 10089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:18.689428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:41961 (local address 127.1.125.254:38731)
0810 07:20:18.689588 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:18.689592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:18.690433 (+ 841us) server_negotiation.cc:407] Connection header received
0810 07:20:18.691746 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:18.691749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:18.691797 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:18.691873 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:18.693190 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.693911 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.695192 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.695434 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.695565 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:18.698271 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:18.698275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:18.698278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:18.698311 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:18.699704 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:18.700321 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:18.700325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:18.700327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:18.700413 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:18.700807 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:18.700810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:18.700812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:18.701003 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:18.701143 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:18.701658 (+ 515us) server_negotiation.cc:299] Negotiation successful
0810 07:20:18.701785 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 07:20:18.702790 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:18.689812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38731 (local address 127.1.125.193:41961)
0810 07:20:18.690267 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:18.690310 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:18.691543 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:18.692040 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:18.692046 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:18.692386 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:18.692987 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:18.693014 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.694039 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.694042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:18.695063 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:18.695073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.695601 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.695606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:18.695779 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:18.696396 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:18.696413 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:18.698093 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:18.699833 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:18.699839 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:18.699853 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:18.700195 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:18.700534 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:18.700538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:18.700540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:18.700701 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:18.701144 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:18.701151 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:18.701405 (+ 254us) client_negotiation.cc:769] Sending connection context
0810 07:20:18.701612 (+ 207us) client_negotiation.cc:240] Negotiation successful
0810 07:20:18.701939 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":131,"threads_started":1}
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:20:18.704104 10317 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38731
I20260810 07:20:18.704380 10317 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:18.704916 10317 heartbeater.cc:511] Master 127.1.125.254:38731 requested a full tablet report, sending...
I20260810 07:20:18.706606 10032 ts_manager.cc:194] Registered new tserver with Master: 40bc4c4f76464bf1a8be37a7eaa0a564 (127.1.125.193:40819)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
I20260810 07:20:18.707908 10032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:41961
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:20:18.723752 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38731
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:43801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:18.838972 10327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:18.839217 10327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:18.839298 10327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:18.843391 10327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:18.843456 10327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:18.843567 10327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:18.848795 10327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:18.849820 10327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:18.850845 10327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:20:18.860910 10327 server_base.cc:1034] running on GCE node
W20260810 07:20:18.860884 10334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:18.860858 10336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:18.860865 10333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:18.861567 10327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:18.862257 10327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:18.863485 10327 hybrid_clock.cc:648] HybridClock initialized: now 1786346418863449 us; error 45 us; skew 500 ppm
Aug 10 07:20:18 dist-test-slave-44d2 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:18.866460 10327 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:20:18.867383 10327 webserver.cc:460] Webserver started at http://127.1.125.194:37937/ using document root <none> and password file <none>
I20260810 07:20:18.867970 10327 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:18.868065 10327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:18.868394 10327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:18.870592 10327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "3037c31c4bd54060958b7cff15fc82e2"
format_stamp: "Formatted at 2026-08-10 07:20:18 on dist-test-slave-44d2"
server_key: "829360be2f3482d5abba63805e220fa1"
server_key_iv: "d832b4eba9b03325c5b3a0f338b132b3"
server_key_version: "encryptionkey@0"
I20260810 07:20:18.871261 10327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "3037c31c4bd54060958b7cff15fc82e2"
format_stamp: "Formatted at 2026-08-10 07:20:18 on dist-test-slave-44d2"
server_key: "829360be2f3482d5abba63805e220fa1"
server_key_iv: "d832b4eba9b03325c5b3a0f338b132b3"
server_key_version: "encryptionkey@0"
I20260810 07:20:18.875100 10327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:20:18.877724 10343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:18.878948 10327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:20:18.879105 10327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "3037c31c4bd54060958b7cff15fc82e2"
format_stamp: "Formatted at 2026-08-10 07:20:18 on dist-test-slave-44d2"
server_key: "829360be2f3482d5abba63805e220fa1"
server_key_iv: "d832b4eba9b03325c5b3a0f338b132b3"
server_key_version: "encryptionkey@0"
I20260810 07:20:18.879220 10327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:18.898520 10327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:18.901935 10327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:18.902155 10327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:18.902782 10327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:18.903642 10327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:18.903715 10327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:18.903786 10327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:18.903841 10327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:18.912914 10327 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:40359
I20260810 07:20:18.912957 10512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:40359 every 8 connection(s)
I20260810 07:20:18.913841 10327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:18.920408 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 10327
I20260810 07:20:18.920593 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:18.920881 1527 external_mini_cluster.cc:1428] Setting key a8b94a94051ea8ff819049aa7408258b
Aug 10 07:20:18 dist-test-slave-44d2 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:18.928736 10089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:18.915657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:33315 (local address 127.1.125.254:38731)
0810 07:20:18.915810 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:18.915814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:18.916689 (+ 875us) server_negotiation.cc:407] Connection header received
0810 07:20:18.918221 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:18.918225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:18.918287 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:18.918395 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:18.919572 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.920112 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.921586 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.921867 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.921974 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:18.924816 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:18.924819 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:18.924821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:18.924847 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:18.926283 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:18.926854 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:18.926858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:18.926859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:18.926995 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:18.927405 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:18.927409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:18.927410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:18.927640 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:18.927749 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:18.928429 (+ 680us) server_negotiation.cc:299] Negotiation successful
0810 07:20:18.928571 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 07:20:18.929602 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:18.916018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38731 (local address 127.1.125.194:33315)
0810 07:20:18.916521 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:18.916586 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:18.917996 (+ 1410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:18.918572 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:18.918585 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:18.918944 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:18.919421 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:18.919434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.920311 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.920318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:18.921450 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:18.921461 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:18.922035 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:18.922038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:18.922253 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:18.922883 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:18.922900 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:18.924644 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:18.926410 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:18.926416 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:18.926429 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:18.926733 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:18.927108 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:18.927111 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:18.927113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:18.927314 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:18.927748 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:18.927754 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:18.928120 (+ 366us) client_negotiation.cc:769] Sending connection context
0810 07:20:18.928337 (+ 217us) client_negotiation.cc:240] Negotiation successful
0810 07:20:18.928745 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":120,"threads_started":1}
I20260810 07:20:18.930831 10513 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38731
I20260810 07:20:18.931097 10513 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:18.931597 10513 heartbeater.cc:511] Master 127.1.125.254:38731 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
I20260810 07:20:18.932760 10032 ts_manager.cc:194] Registered new tserver with Master: 3037c31c4bd54060958b7cff15fc82e2 (127.1.125.194:40359)
I20260810 07:20:18.933341 10032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:33315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:20:18.952822 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38731
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:43801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:19.068838 10520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:19.069094 10520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:19.069136 10520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:19.073403 10520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:19.073462 10520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:19.073570 10520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:19.078853 10520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:19.079926 10520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:19.081028 10520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:19.090972 10530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:19.090951 10528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:19.090953 10520 server_base.cc:1034] running on GCE node
W20260810 07:20:19.090934 10527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:19.091614 10520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:19.092272 10520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:19.093470 10520 hybrid_clock.cc:648] HybridClock initialized: now 1786346419093482 us; error 71 us; skew 500 ppm
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346419, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:19.096200 10520 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:20:19.097116 10520 webserver.cc:460] Webserver started at http://127.1.125.195:43095/ using document root <none> and password file <none>
I20260810 07:20:19.097697 10520 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:19.097792 10520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:19.098032 10520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:19.100131 10520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "0d808c6c900a4388aff739b0139c2bc5"
format_stamp: "Formatted at 2026-08-10 07:20:19 on dist-test-slave-44d2"
server_key: "7d9726036a6ab70b989edb54688fa168"
server_key_iv: "6e6c3a17505465c4cc8f284129beaace"
server_key_version: "encryptionkey@0"
I20260810 07:20:19.100754 10520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "0d808c6c900a4388aff739b0139c2bc5"
format_stamp: "Formatted at 2026-08-10 07:20:19 on dist-test-slave-44d2"
server_key: "7d9726036a6ab70b989edb54688fa168"
server_key_iv: "6e6c3a17505465c4cc8f284129beaace"
server_key_version: "encryptionkey@0"
I20260810 07:20:19.104553 10520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 07:20:19.107002 10541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:19.108242 10520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:19.108382 10520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "0d808c6c900a4388aff739b0139c2bc5"
format_stamp: "Formatted at 2026-08-10 07:20:19 on dist-test-slave-44d2"
server_key: "7d9726036a6ab70b989edb54688fa168"
server_key_iv: "6e6c3a17505465c4cc8f284129beaace"
server_key_version: "encryptionkey@0"
I20260810 07:20:19.108525 10520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:19.131487 10520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:19.134454 10520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:19.134673 10520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:19.135305 10520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:19.136183 10520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:19.136257 10520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:19.136335 10520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:19.136379 10520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:19.146034 10520 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:36273
I20260810 07:20:19.146056 10705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:36273 every 8 connection(s)
I20260810 07:20:19.146991 10520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:19.150357 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 10520
I20260810 07:20:19.150465 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:19.150779 1527 external_mini_cluster.cc:1428] Setting key 57bd0c2940409d21b2b4f17e42a58b42
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346419, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:19.161202 10089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:19.148942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:33953 (local address 127.1.125.254:38731)
0810 07:20:19.149094 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:19.149099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:19.150024 (+ 925us) server_negotiation.cc:407] Connection header received
0810 07:20:19.151653 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:19.151657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:19.151716 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:19.151802 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:19.153154 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.153709 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.154892 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.155124 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.155240 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:19.157796 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:19.157800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:19.157803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:19.157836 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:19.159231 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:19.159712 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:19.159715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:19.159717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:19.159822 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:19.160182 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:19.160185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:19.160186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:19.160370 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:19.160453 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:19.160963 (+ 510us) server_negotiation.cc:299] Negotiation successful
0810 07:20:19.161078 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:20:19.162035 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:19.149398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38731 (local address 127.1.125.195:33953)
0810 07:20:19.149865 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:19.149911 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:19.151435 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:19.151994 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:19.152001 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:19.152382 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:19.153005 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:19.153020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.153855 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.153858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:19.154762 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:19.154772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.155282 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.155287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:19.155461 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:19.156014 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:19.156030 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:19.157631 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:19.159337 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:19.159342 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:19.159353 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:19.159622 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:19.159921 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:19.159923 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:19.159924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:19.160063 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:19.160467 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:19.160471 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:19.160741 (+ 270us) client_negotiation.cc:769] Sending connection context
0810 07:20:19.160902 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 07:20:19.161155 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":127,"threads_started":1}
I20260810 07:20:19.163203 10706 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38731
I20260810 07:20:19.163507 10706 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:19.164124 10706 heartbeater.cc:511] Master 127.1.125.254:38731 requested a full tablet report, sending...
I20260810 07:20:19.165330 10032 ts_manager.cc:194] Registered new tserver with Master: 0d808c6c900a4388aff739b0139c2bc5 (127.1.125.195:36273)
I20260810 07:20:19.165896 10032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:33953
I20260810 07:20:19.177021 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/test-admin.keytab.
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:19.196630 1527 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 07:20:19.199604 10718 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:19.198143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34156 (local address 127.1.125.193:40819)
0810 07:20:19.198493 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:19.198510 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:19.198536 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:20:19.198601 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:19.198607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:19.198804 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:19.198931 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:19.199360 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34156: BlockingRecv error: recv got EOF from 127.0.0.1:34156 (error 108)
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":117,"threads_started":1}
W20260810 07:20:19.199719 1527 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/krb5cc' of type FILE
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:19.200834 1527 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:19.201804 1527 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.193@KRBTEST.COM
I20260810 07:20:19.211310 10718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:19.202740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34160 (local address 127.1.125.193:40819)
0810 07:20:19.202907 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:19.202911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:19.202975 (+ 64us) server_negotiation.cc:407] Connection header received
0810 07:20:19.203071 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:19.203073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:19.203115 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:19.203185 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:19.203696 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.204447 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.205326 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.205711 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.205795 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:19.207947 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:19.207957 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:19.207971 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:19.208002 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:19.209726 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:19.210113 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:19.210118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:19.210120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:19.210197 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:19.210427 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:19.210430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:19.210431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:19.210667 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:19.210817 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:19.210973 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 07:20:19.211160 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260810 07:20:19.215329 10718 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:19.214372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34174 (local address 127.1.125.193:40819)
0810 07:20:19.214494 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:19.214498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:19.214619 (+ 121us) server_negotiation.cc:407] Connection header received
0810 07:20:19.214712 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:19.214714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:19.214748 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:19.214810 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:19.215200 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34174: BlockingRecv error: recv got EOF from 127.0.0.1:34174 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 07:20:19.215428 1527 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/krb5cc' of type FILE
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:19.216454 1527 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:19.217465 1527 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.193@KRBTEST.COM
I20260810 07:20:19.225677 10718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:19.218290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34188 (local address 127.1.125.193:40819)
0810 07:20:19.218411 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:19.218417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:19.218522 (+ 105us) server_negotiation.cc:407] Connection header received
0810 07:20:19.218613 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:19.218615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:19.218648 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:19.218709 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:19.219148 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.219627 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.220341 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.220628 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.220706 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:19.222839 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:19.222842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:19.222844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:19.222865 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:19.224269 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:19.224690 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:19.224693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:19.224694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:19.224764 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:19.224971 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:19.224972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:19.224973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:19.225122 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:19.225212 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:19.225373 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 07:20:19.225457 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 07:20:19.228281 10718 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:19.227345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34202 (local address 127.1.125.193:40819)
0810 07:20:19.227484 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:19.227487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:19.227499 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:19.227598 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:19.227600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:19.227655 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:19.227710 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:19.228175 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34202: BlockingRecv error: recv got EOF from 127.0.0.1:34202 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 07:20:19.228344 1527 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346386457763-1527-0/krb5kdc/krb5cc' of type FILE
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:19.229611 1527 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:19.230551 1527 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.193@KRBTEST.COM
I20260810 07:20:19.239316 10718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:19.231358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34208 (local address 127.1.125.193:40819)
0810 07:20:19.231491 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:19.231496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:19.231603 (+ 107us) server_negotiation.cc:407] Connection header received
0810 07:20:19.231718 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:19.231721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:19.231758 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:19.231848 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:19.232348 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.232919 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.233711 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:19.233937 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:19.234049 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:19.236234 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:19.236238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:19.236240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:19.236265 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:19.237706 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:19.238132 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:19.238136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:19.238138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:19.238222 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:19.238510 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:19.238513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:19.238515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:19.238674 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:19.238820 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:19.239069 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 07:20:19.239177 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:20:19.240267 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 10132
I20260810 07:20:19.254385 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 10327
I20260810 07:20:19.271679 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 10520
I20260810 07:20:19.285483 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 9988
2026-08-10T07:20:19Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2565 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[10738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[10738](info): set up 2 sockets
Aug 10 07:20:19 dist-test-slave-44d2 krb5kdc[10738](info): commencing operation
krb5kdc: starting...
W20260810 07:20:20.633438 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:20:20 dist-test-slave-44d2 krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:20Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.254.keytab.
Entry for principal oryx/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.254.keytab.
I20260810 07:20:20.763814 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:45317
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.254.keytab
--principal=oryx/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:45317
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:20.879238 10758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:20.879537 10758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:20.879626 10758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:20.883663 10758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:20.883750 10758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:20.883788 10758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:20.883845 10758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:20.883883 10758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:20.890821 10758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:45317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.254.keytab
--principal=oryx/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:45317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:20.892300 10758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:20.893723 10758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:20.903474 10767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:20.903462 10766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:20.903545 10769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:20.903993 10758 server_base.cc:1034] running on GCE node
I20260810 07:20:20.904696 10758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:20.905822 10758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:20.907029 10758 hybrid_clock.cc:648] HybridClock initialized: now 1786346420906999 us; error 53 us; skew 500 ppm
Aug 10 07:20:20 dist-test-slave-44d2 krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346420, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:20.910203 10758 init.cc:377] Logged in from keytab as oryx/127.1.125.254@KRBTEST.COM (short username oryx)
I20260810 07:20:20.911165 10758 webserver.cc:460] Webserver started at http://127.1.125.254:37045/ using document root <none> and password file <none>
I20260810 07:20:20.911813 10758 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:20.911906 10758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:20.912155 10758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:20.914400 10758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "94a50c31fbb042299f5f43ac5b7c8755"
format_stamp: "Formatted at 2026-08-10 07:20:20 on dist-test-slave-44d2"
server_key: "d4890092fa75ea4d0b99cf692487fd06"
server_key_iv: "831cd1e553a10e39afa1ce9eb33135b3"
server_key_version: "encryptionkey@0"
I20260810 07:20:20.914987 10758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "94a50c31fbb042299f5f43ac5b7c8755"
format_stamp: "Formatted at 2026-08-10 07:20:20 on dist-test-slave-44d2"
server_key: "d4890092fa75ea4d0b99cf692487fd06"
server_key_iv: "831cd1e553a10e39afa1ce9eb33135b3"
server_key_version: "encryptionkey@0"
I20260810 07:20:20.918885 10758 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:20.921484 10779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:20.922811 10758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:20.922961 10758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "94a50c31fbb042299f5f43ac5b7c8755"
format_stamp: "Formatted at 2026-08-10 07:20:20 on dist-test-slave-44d2"
server_key: "d4890092fa75ea4d0b99cf692487fd06"
server_key_iv: "831cd1e553a10e39afa1ce9eb33135b3"
server_key_version: "encryptionkey@0"
I20260810 07:20:20.923090 10758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:20.948091 10758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:20.951431 10758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:20.951651 10758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:20.959846 10758 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:45317
I20260810 07:20:20.960273 10851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:45317 every 8 connection(s)
I20260810 07:20:20.960933 10758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:20.963897 10853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:20.963943 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 10758
I20260810 07:20:20.964111 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:20.964512 1527 external_mini_cluster.cc:1428] Setting key fea32ab8d05fc06721b3e5430eadd72c
I20260810 07:20:20.971202 10853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755: Bootstrap starting.
I20260810 07:20:20.973728 10853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:20.974637 10853 log.cc:824] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755: Log is configured to *not* fsync() on all Append() calls
Aug 10 07:20:20 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:20.976775 10853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755: No bootstrap required, opened a new log
I20260810 07:20:20.981292 10853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } }
I20260810 07:20:20.981637 10853 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:20.981696 10853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94a50c31fbb042299f5f43ac5b7c8755, State: Initialized, Role: FOLLOWER
I20260810 07:20:20.982421 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:20.966194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44858 (local address 127.1.125.254:45317)
0810 07:20:20.966705 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:20.966723 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:20.966769 (+ 46us) server_negotiation.cc:407] Connection header received
0810 07:20:20.967537 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:20.967562 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:20.968005 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:20.968366 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:20.969526 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.971008 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.972391 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:20.973123 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:20.973284 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:20.976532 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:20.976539 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:20.976556 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:20.976586 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:20.978677 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:20.979211 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:20.979216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:20.979222 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:20.979351 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:20.979645 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:20.979648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:20.979649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:20.980139 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:20.980999 (+ 860us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:20.981146 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 07:20:20.981454 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":151,"threads_started":1}
I20260810 07:20:20.982353 10853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } }
I20260810 07:20:20.982730 10853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:20.982854 10853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:20.983008 10853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:20.984889 10853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } }
I20260810 07:20:20.985284 10853 leader_election.cc:302] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94a50c31fbb042299f5f43ac5b7c8755; no voters:
I20260810 07:20:20.985608 10853 leader_election.cc:288] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:20.985770 10861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:20.986011 10861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [term 1 LEADER]: Becoming Leader. State: Replica: 94a50c31fbb042299f5f43ac5b7c8755, State: Running, Role: LEADER
I20260810 07:20:20.986320 10861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } }
I20260810 07:20:20.987221 10853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:20.988181 10863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } }
I20260810 07:20:20.988350 10863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:20.988181 10864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94a50c31fbb042299f5f43ac5b7c8755. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } }
I20260810 07:20:20.988713 10864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: This master's current role is: LEADER
W20260810 07:20:20.992995 10886 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:20:20.993088 10886 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:20:20.993166 10875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:20.993866 10875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:21.000026 10875 catalog_manager.cc:1347] Generated new cluster ID: da4a4ebf438b41e1bb371235fd054f0f
I20260810 07:20:21.000118 10875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:21.017954 10875 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:21.019232 10875 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:21.025965 10875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755: Generated new TSK 0
I20260810 07:20:21.026816 10875 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab.
Entry for principal oryx/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab.
I20260810 07:20:21.063791 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45317
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab
--principal=oryx/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:21.179325 10896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:21.179580 10896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:21.179623 10896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:21.183616 10896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:21.183670 10896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:21.183727 10896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:21.189230 10896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab
--principal=oryx/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:21.190332 10896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:21.191357 10896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:21.201284 10906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:21.201292 10909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:21.201308 10904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:21.201777 10896 server_base.cc:1034] running on GCE node
I20260810 07:20:21.202176 10896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:21.202853 10896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:21.204052 10896 hybrid_clock.cc:648] HybridClock initialized: now 1786346421204036 us; error 40 us; skew 500 ppm
Aug 10 07:20:21 dist-test-slave-44d2 krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346421, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:21.206892 10896 init.cc:377] Logged in from keytab as oryx/127.1.125.193@KRBTEST.COM (short username oryx)
I20260810 07:20:21.207842 10896 webserver.cc:460] Webserver started at http://127.1.125.193:33279/ using document root <none> and password file <none>
I20260810 07:20:21.208436 10896 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:21.208559 10896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:21.208809 10896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:21.210898 10896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "06f27bfb4699425ead7908ac41157923"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "7ff91d5028c8fa3fb914921ab423ab82"
server_key_iv: "bcf76f7a5eef65444ae324683bd6b370"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.211480 10896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "06f27bfb4699425ead7908ac41157923"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "7ff91d5028c8fa3fb914921ab423ab82"
server_key_iv: "bcf76f7a5eef65444ae324683bd6b370"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.215874 10896 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260810 07:20:21.218410 10923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.219630 10896 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:20:21.219774 10896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "06f27bfb4699425ead7908ac41157923"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "7ff91d5028c8fa3fb914921ab423ab82"
server_key_iv: "bcf76f7a5eef65444ae324683bd6b370"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.219913 10896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:21.236507 10896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:21.239292 10896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:21.239498 10896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:21.240145 10896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:21.241029 10896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:21.241101 10896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.241174 10896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:21.241218 10896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.250439 10896 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:33107
I20260810 07:20:21.250586 11073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:33107 every 8 connection(s)
I20260810 07:20:21.251380 10896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:21.260728 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 10896
I20260810 07:20:21.260882 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:21.261194 1527 external_mini_cluster.cc:1428] Setting key 55d3377a02e2d015933eb8309e0981a8
Aug 10 07:20:21 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346421, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:21.266435 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.253325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:46813 (local address 127.1.125.254:45317)
0810 07:20:21.253471 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:21.253475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:21.254566 (+ 1091us) server_negotiation.cc:407] Connection header received
0810 07:20:21.255543 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:21.255547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:21.255611 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:21.255695 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:21.256990 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.257618 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.258880 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.259127 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.259247 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.262195 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:21.262199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:21.262201 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:21.262227 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:21.263845 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:21.264430 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:21.264440 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:21.264442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:21.264609 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:21.265055 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:21.265076 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:21.265080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:21.265348 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:21.265462 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:21.266197 (+ 735us) server_negotiation.cc:299] Negotiation successful
0810 07:20:21.266310 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:20:21.267833 11079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.253707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.193:46813)
0810 07:20:21.254406 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:21.254448 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:21.255323 (+ 875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:21.255874 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:21.255881 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:21.256253 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:21.256830 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.256844 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.257803 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.257808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.258741 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.258752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.259294 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.259298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.259481 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.260136 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:21.260154 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:21.262009 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:21.263973 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:21.263979 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:21.263994 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:21.264314 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:21.264711 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:21.264715 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:21.264717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:21.264876 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:21.265528 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:21.265538 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:21.265864 (+ 326us) client_negotiation.cc:769] Sending connection context
0810 07:20:21.266140 (+ 276us) client_negotiation.cc:240] Negotiation successful
0810 07:20:21.266661 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":113,"threads_started":1}
I20260810 07:20:21.269376 11077 heartbeater.cc:347] Connected to a master server at 127.1.125.254:45317
I20260810 07:20:21.269654 11077 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:21.270206 11077 heartbeater.cc:511] Master 127.1.125.254:45317 requested a full tablet report, sending...
I20260810 07:20:21.271852 10804 ts_manager.cc:194] Registered new tserver with Master: 06f27bfb4699425ead7908ac41157923 (127.1.125.193:33107)
WARNING: no policy specified for oryx/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:20:21.273159 10804 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:46813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab.
Entry for principal oryx/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab.
I20260810 07:20:21.291610 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45317
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab
--principal=oryx/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:21.405932 11082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:21.406177 11082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:21.406219 11082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:21.410121 11082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:21.410171 11082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:21.410279 11082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:21.415362 11082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab
--principal=oryx/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:21.416355 11082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:21.417377 11082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:21.427337 11088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:21.427384 11094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:21.427394 11090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:21.427568 11082 server_base.cc:1034] running on GCE node
I20260810 07:20:21.428042 11082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:21.428757 11082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:21.429949 11082 hybrid_clock.cc:648] HybridClock initialized: now 1786346421429922 us; error 38 us; skew 500 ppm
Aug 10 07:20:21 dist-test-slave-44d2 krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346421, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:21.432808 11082 init.cc:377] Logged in from keytab as oryx/127.1.125.194@KRBTEST.COM (short username oryx)
I20260810 07:20:21.433610 11082 webserver.cc:460] Webserver started at http://127.1.125.194:35477/ using document root <none> and password file <none>
I20260810 07:20:21.434185 11082 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:21.434252 11082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:21.434423 11082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:21.436393 11082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "ce643b0a17074c9e9a974c4ac36f711e"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "b24ee46815ccbef74af851d7f880017d"
server_key_iv: "855cfe9dfd5f435e972c13a2cab5fbef"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.436986 11082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "ce643b0a17074c9e9a974c4ac36f711e"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "b24ee46815ccbef74af851d7f880017d"
server_key_iv: "855cfe9dfd5f435e972c13a2cab5fbef"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.440755 11082 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:21.443087 11108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.444283 11082 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:21.444381 11082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "ce643b0a17074c9e9a974c4ac36f711e"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "b24ee46815ccbef74af851d7f880017d"
server_key_iv: "855cfe9dfd5f435e972c13a2cab5fbef"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.444464 11082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:21.462694 11082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:21.466226 11082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:21.466391 11082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:21.466952 11082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:21.467777 11082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:21.467834 11082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.467876 11082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:21.467897 11082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.477293 11082 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:33527
I20260810 07:20:21.477325 11288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:33527 every 8 connection(s)
I20260810 07:20:21.478196 11082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:20:21 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346421, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:21.488168 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 11082
I20260810 07:20:21.488286 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:21.488696 1527 external_mini_cluster.cc:1428] Setting key 9864ce423fe694dd60d27bfdd2aa2b57
I20260810 07:20:21.492995 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.480006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:49609 (local address 127.1.125.254:45317)
0810 07:20:21.480152 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:21.480163 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:21.481393 (+ 1230us) server_negotiation.cc:407] Connection header received
0810 07:20:21.482333 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:21.482337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:21.482402 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:21.482481 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:21.483679 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.484265 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.485520 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.485777 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.485896 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.488810 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:21.488817 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:21.488824 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:21.488856 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:21.490519 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:21.491184 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:21.491187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:21.491188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:21.491261 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:21.491658 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:21.491660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:21.491661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:21.491824 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:21.491899 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:21.492620 (+ 721us) server_negotiation.cc:299] Negotiation successful
0810 07:20:21.492774 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:20:21.493772 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.480337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.194:49609)
0810 07:20:21.481200 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:21.481265 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:21.482110 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:21.482660 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:21.482667 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:21.483026 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:21.483522 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.483535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.484419 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.484423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.485386 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.485397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.485938 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.485943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.486123 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.486754 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:21.486771 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:21.488598 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:21.490637 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:21.490643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:21.490658 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:21.491077 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:21.491350 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:21.491356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:21.491362 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:21.491556 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:21.491928 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:21.491936 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:21.492268 (+ 332us) client_negotiation.cc:769] Sending connection context
0810 07:20:21.492474 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 07:20:21.492887 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":209,"threads_started":1}
I20260810 07:20:21.495014 11289 heartbeater.cc:347] Connected to a master server at 127.1.125.254:45317
I20260810 07:20:21.495270 11289 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:21.495774 11289 heartbeater.cc:511] Master 127.1.125.254:45317 requested a full tablet report, sending...
I20260810 07:20:21.496963 10804 ts_manager.cc:194] Registered new tserver with Master: ce643b0a17074c9e9a974c4ac36f711e (127.1.125.194:33527)
I20260810 07:20:21.497864 10804 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:49609
WARNING: no policy specified for oryx/127.1.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab.
Entry for principal oryx/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab.
I20260810 07:20:21.519224 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45317
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab
--principal=oryx/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:21.638576 11297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:21.638854 11297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:21.638901 11297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:21.642912 11297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:21.642976 11297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:21.643054 11297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:21.648470 11297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab
--principal=oryx/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:21.649466 11297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:21.650450 11297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:21.660516 11304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:21.660516 11309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:21.660693 11297 server_base.cc:1034] running on GCE node
W20260810 07:20:21.660848 11307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:21.661234 11297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:21.661865 11297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:21.663034 11297 hybrid_clock.cc:648] HybridClock initialized: now 1786346421663016 us; error 29 us; skew 500 ppm
Aug 10 07:20:21 dist-test-slave-44d2 krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346421, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:21.665884 11297 init.cc:377] Logged in from keytab as oryx/127.1.125.195@KRBTEST.COM (short username oryx)
I20260810 07:20:21.666738 11297 webserver.cc:460] Webserver started at http://127.1.125.195:41509/ using document root <none> and password file <none>
I20260810 07:20:21.667259 11297 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:21.667317 11297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:21.667481 11297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:21.669605 11297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "3f1bb10376c747af8a925456ebb1c4a8"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "a91ec6ec77622ebb7a676da5bf845e31"
server_key_iv: "cb917c645f734c5e21aa7c82803cb730"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.670150 11297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "3f1bb10376c747af8a925456ebb1c4a8"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "a91ec6ec77622ebb7a676da5bf845e31"
server_key_iv: "cb917c645f734c5e21aa7c82803cb730"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.674214 11297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:21.676702 11319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.677938 11297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:21.678071 11297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "3f1bb10376c747af8a925456ebb1c4a8"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "a91ec6ec77622ebb7a676da5bf845e31"
server_key_iv: "cb917c645f734c5e21aa7c82803cb730"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.678189 11297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:21.702564 11297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:21.706063 11297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:21.706279 11297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:21.706911 11297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:21.707767 11297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:21.707818 11297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.707888 11297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:21.707927 11297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.717499 11297 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:44005
I20260810 07:20:21.717554 11485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:44005 every 8 connection(s)
I20260810 07:20:21.718505 11297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:21.726120 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 11297
I20260810 07:20:21.726269 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:21.726559 1527 external_mini_cluster.cc:1428] Setting key 8334ecc65d480491504d478f95ae741b
Aug 10 07:20:21 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346421, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:21.732225 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.720349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:53443 (local address 127.1.125.254:45317)
0810 07:20:21.720499 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:21.720505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:21.721491 (+ 986us) server_negotiation.cc:407] Connection header received
0810 07:20:21.722414 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:21.722419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:21.722481 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:21.722584 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:21.723826 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.724376 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.725657 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.725887 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.725995 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.729130 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:21.729137 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:21.729140 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:21.729169 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:21.730472 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:21.730942 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:21.730945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:21.730946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:21.731018 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:21.731313 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:21.731315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:21.731316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:21.731482 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:21.731556 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:21.732002 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 07:20:21.732104 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:20:21.733155 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.720683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.195:53443)
0810 07:20:21.721337 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:21.721379 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:21.722198 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:21.722770 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:21.722777 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:21.723128 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:21.723679 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.723695 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.724535 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.724539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.725508 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.725525 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.726051 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.726055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.726229 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.726965 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:21.726986 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:21.728892 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:21.730577 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:21.730582 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:21.730594 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:21.730848 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:21.731095 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:21.731097 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:21.731099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:21.731229 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:21.731567 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:21.731571 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:21.731791 (+ 220us) client_negotiation.cc:769] Sending connection context
0810 07:20:21.731938 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 07:20:21.732189 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":120,"threads_started":1}
I20260810 07:20:21.734180 11486 heartbeater.cc:347] Connected to a master server at 127.1.125.254:45317
I20260810 07:20:21.734426 11486 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:21.734906 11486 heartbeater.cc:511] Master 127.1.125.254:45317 requested a full tablet report, sending...
I20260810 07:20:21.735885 10804 ts_manager.cc:194] Registered new tserver with Master: 3f1bb10376c747af8a925456ebb1c4a8 (127.1.125.195:44005)
I20260810 07:20:21.736454 10804 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:53443
I20260810 07:20:21.741040 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.125.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.125.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.125.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.253.keytab.
Entry for principal oryx/127.1.125.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.253.keytab.
I20260810 07:20:21.771095 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.253:34723
--webserver_interface=127.1.125.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.253.keytab
--principal=oryx/127.1.125.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:45317,127.1.125.253:34723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:21.887336 11491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:21.887614 11491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:21.887691 11491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:21.891865 11491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:21.891938 11491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:21.891957 11491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:21.891974 11491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:21.892030 11491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:21.897330 11491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:45317,127.1.125.253:34723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.253.keytab
--principal=oryx/127.1.125.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.253:34723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.125.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
W20260810 07:20:21.897578 11491 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.125.254:45317,127.1.125.253:34723'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 07:20:21.898511 11491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:21.899603 11491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:21.908473 11499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:21.908514 11501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:21.908653 11491 server_base.cc:1034] running on GCE node
W20260810 07:20:21.908514 11498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:21.909332 11491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:21.910502 11491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:21.911700 11491 hybrid_clock.cc:648] HybridClock initialized: now 1786346421911666 us; error 46 us; skew 500 ppm
Aug 10 07:20:21 dist-test-slave-44d2 krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346421, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:21.914568 11491 init.cc:377] Logged in from keytab as oryx/127.1.125.253@KRBTEST.COM (short username oryx)
I20260810 07:20:21.915467 11491 webserver.cc:460] Webserver started at http://127.1.125.253:35193/ using document root <none> and password file <none>
I20260810 07:20:21.916057 11491 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:21.916169 11491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:21.916415 11491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:21.918627 11491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/data/instance:
uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "d6d674b8586e90b0b984f1295f316daa"
server_key_iv: "67afd732e5f6dde71de5b968e8719107"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.919219 11491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/wal/instance:
uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "d6d674b8586e90b0b984f1295f316daa"
server_key_iv: "67afd732e5f6dde71de5b968e8719107"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.923255 11491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:20:21.926013 11514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:21.927624 11491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 07:20:21.927774 11491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/wal
uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "d6d674b8586e90b0b984f1295f316daa"
server_key_iv: "67afd732e5f6dde71de5b968e8719107"
server_key_version: "encryptionkey@0"
I20260810 07:20:21.927892 11491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:21.941713 11491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:21.944626 11491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:21.944831 11491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:21.952425 11491 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.253:34723
I20260810 07:20:21.952734 11591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.253:34723 every 8 connection(s)
I20260810 07:20:21.953542 11491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/data/info.pb
I20260810 07:20:21.956565 11594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:21.958474 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 11491
I20260810 07:20:21.961283 11594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:20:21 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.125.253@KRBTEST.COM
Aug 10 07:20:21 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346421, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.253@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:21.975059 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.963542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44868 (local address 127.1.125.254:45317)
0810 07:20:21.963698 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:21.963705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:21.964066 (+ 361us) server_negotiation.cc:407] Connection header received
0810 07:20:21.964267 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:21.964272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:21.964338 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:21.964383 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:21.965292 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.966082 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.967327 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.967663 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.967774 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.971095 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:21.971106 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:21.971109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:21.971133 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:21.972915 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:21.973483 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:21.973487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:21.973489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:21.973588 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:21.973954 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:21.973958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:21.973959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:21.974213 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:21.974321 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:21.974767 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 07:20:21.974883 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 07:20:21.975807 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.960303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46092 (local address 127.1.125.253:34723)
0810 07:20:21.960812 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:21.960831 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:21.960870 (+ 39us) server_negotiation.cc:407] Connection header received
0810 07:20:21.961534 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:21.961552 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:21.961919 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:21.962319 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:21.963259 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.964864 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.965952 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.966690 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.966848 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.970537 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:21.970547 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:21.970561 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:21.970589 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:21.973004 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:21.973480 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:21.973486 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:21.973487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:21.973610 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:21.973899 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:21.973903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:21.973905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:21.974375 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:21.974613 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:21.975256 (+ 643us) server_negotiation.cc:299] Negotiation successful
0810 07:20:21.975391 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":346,"thread_start_us":194,"threads_started":1}
I20260810 07:20:21.975807 11599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.963521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:44868)
0810 07:20:21.963939 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:21.963988 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:21.964154 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:21.964607 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:21.964611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:21.964974 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:21.965145 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.965155 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.966221 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.966224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.967196 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.967210 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.967860 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.967864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.967989 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.968707 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:21.968727 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:21.970940 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:21.973040 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:21.973050 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:21.973059 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:21.973360 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:21.973695 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:21.973698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:21.973700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:21.973833 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:21.974337 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:21.974343 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:21.974516 (+ 173us) client_negotiation.cc:769] Sending connection context
0810 07:20:21.974710 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 07:20:21.974952 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":1,"thread_start_us":167,"threads_started":1}
I20260810 07:20:21.977348 11594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:20:21 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346421, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.253@KRBTEST.COM for oryx/127.1.125.253@KRBTEST.COM
I20260810 07:20:21.985669 11599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.977853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:34723 (local address 127.0.0.1:46096)
0810 07:20:21.977977 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:21.977992 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:21.978073 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:21.978300 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:21.978303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:21.978463 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:21.978583 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.978589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.979775 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.979778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.980278 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:21.980286 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.980825 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.980828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:21.980924 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.981397 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:21.981409 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:21.982757 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:21.984211 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:21.984213 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:21.984215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:21.984396 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:21.984749 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:21.984755 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:21.984760 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:21.984854 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:21.985219 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:21.985221 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:21.985296 (+ 75us) client_negotiation.cc:769] Sending connection context
0810 07:20:21.985378 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 07:20:21.985472 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 07:20:21.985669 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:21.977943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46096 (local address 127.1.125.253:34723)
0810 07:20:21.978112 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:21.978117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:21.978129 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:21.978163 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:21.978165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:21.978208 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:21.978291 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:21.978696 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.979625 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.980398 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:21.980716 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:21.980826 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:21.982877 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:21.982880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:21.982882 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:21.982903 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:21.984112 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:21.984555 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:21.984559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:21.984560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:21.984646 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:21.984952 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:21.984954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:21.984955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:21.985132 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:21.985208 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:21.985437 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 07:20:21.985526 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 07:20:21.988845 11594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: Bootstrap starting.
I20260810 07:20:21.991142 11594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:21.992085 11594 log.cc:824] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:21.994155 11594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: No bootstrap required, opened a new log
I20260810 07:20:21.997228 11594 raft_consensus.cc:348] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } }
I20260810 07:20:21.997447 11594 raft_consensus.cc:374] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:21.997526 11594 raft_consensus.cc:729] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e62a6dfd26a74cb7ba4d14bac5f9c360, State: Initialized, Role: FOLLOWER
I20260810 07:20:21.997960 11594 consensus_queue.cc:260] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } }
I20260810 07:20:21.998513 11601 sys_catalog.cc:455] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } } }
I20260810 07:20:21.998668 11601 sys_catalog.cc:458] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:20:21.999168 11594 sys_catalog.cc:564] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 07:20:22.004175 11614 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:20:22.004302 11614 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:20:22.006569 11491 master_runner.cc:428] Detected that this master e62a6dfd26a74cb7ba4d14bac5f9c360 is joining an existing cluster
I20260810 07:20:22.006620 11491 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.125.253:34723
I20260810 07:20:22.017969 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.009280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.1.125.253:34723)
0810 07:20:22.009378 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.009382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.009393 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:20:22.009422 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.009425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.009461 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.009528 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.010185 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.010903 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.012106 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.012387 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.012536 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.013958 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.013962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.013964 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.013987 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.016197 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.016713 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.016717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.016719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.016803 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.017091 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.017094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.017096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.017294 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.017420 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.017673 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.017831 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":39}
I20260810 07:20:22.017982 11626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.008660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:34723 (local address 127.0.0.1:46112)
0810 07:20:22.009121 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.009136 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.009225 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.009625 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.009628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.009892 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.010068 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.010076 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.011044 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.011048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.011959 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.011970 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.012556 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.012559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.012696 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.013241 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.013263 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.013826 (+ 563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.016315 (+ 2489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.016321 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.016325 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.016598 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.016899 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.016902 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.016904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.016988 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.017389 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.017394 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.017504 (+ 110us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.017615 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.017768 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":47,"thread_start_us":126,"threads_started":1}
I20260810 07:20:22.019435 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.009145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.1.125.254:45317)
0810 07:20:22.009263 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.009268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.009282 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:20:22.009325 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.009328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.009384 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.009477 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.010045 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.010905 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.013289 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.013633 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.013748 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.015231 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.015235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.015238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.015265 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.017029 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.017561 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.017565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.017567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.017666 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.017978 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.017982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.017984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.018197 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.018266 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.019149 (+ 883us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.019272 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:20:22.019964 11625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.008626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:44884)
0810 07:20:22.008965 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.008982 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.009081 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.009496 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.009499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.009780 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.009924 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.009931 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.012129 (+ 2198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.012132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.013142 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.013153 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.013773 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.013777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.013875 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.014426 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.014442 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.015070 (+ 628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.017160 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.017164 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.017166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.017445 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.017785 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.017788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.017790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.017879 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.018847 (+ 968us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.018850 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.018942 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.019643 (+ 701us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.019785 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":3,"thread_start_us":175,"threads_started":1}
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346420, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.254@KRBTEST.COM for oryx/127.1.125.253@KRBTEST.COM
I20260810 07:20:22.035554 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.024841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46122 (local address 127.1.125.253:34723)
0810 07:20:22.024987 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.024991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.025868 (+ 877us) server_negotiation.cc:407] Connection header received
0810 07:20:22.026008 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.026010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.026088 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.026177 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.026949 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.027480 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.028759 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.029051 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.029156 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.032088 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.032095 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.032101 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.032133 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.033761 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.034254 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.034258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.034260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.034345 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.034645 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.034648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.034650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.034826 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.034935 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.035283 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.035399 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 07:20:22.035590 11629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.024768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:34723 (local address 127.0.0.1:46122)
0810 07:20:22.025699 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.025750 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.025880 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.026233 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.026240 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.026705 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.026839 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.026847 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.027586 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.027589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.028633 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.028644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.029174 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.029178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.029292 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.029938 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.029953 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.031937 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.033881 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.033893 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.033900 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.034147 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.034449 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.034452 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.034454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.034549 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.034949 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.034957 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.035077 (+ 120us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.035225 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.035416 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":197,"threads_started":1}
I20260810 07:20:22.037168 10804 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.125.253:34723: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360"
member_type: NON_VOTER
last_known_addr {
host: "127.1.125.253"
port: 34723
}
attrs {
promote: true
}
}
dest_uuid: "94a50c31fbb042299f5f43ac5b7c8755"
cas_config_opid_index: -1
I20260810 07:20:22.037887 10804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true } }
I20260810 07:20:22.040086 11634 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e62a6dfd26a74cb7ba4d14bac5f9c360 (127.1.125.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true } } }
I20260810 07:20:22.040761 11554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:22.041728 11634 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.125.253:34723
I20260810 07:20:22.042207 11554 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 FOLLOWER]: Refusing update from remote peer 94a50c31fbb042299f5f43ac5b7c8755: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 07:20:22.042261 11491 master.cc:537] Master@127.1.125.253:34723 shutting down...
I20260810 07:20:22.043051 11637 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:22.044018 11633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true } } }
I20260810 07:20:22.045776 11633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: This master's current role is: LEADER
W20260810 07:20:22.046571 10782 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260810 07:20:22.047470 11491 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:22.047755 11491 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:22.047855 11491 tablet_replica.cc:330] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: stopping tablet replica
W20260810 07:20:22.047894 10782 consensus_peers.cc:487] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 -> Peer e62a6dfd26a74cb7ba4d14bac5f9c360 (127.1.125.253:34723): Couldn't send request to peer e62a6dfd26a74cb7ba4d14bac5f9c360. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
WARNING: no policy specified for oryx/127.1.125.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.125.252@KRBTEST.COM" created.
I20260810 07:20:22.062794 11491 master.cc:552] Master@127.1.125.253:34723 shutdown complete.
I20260810 07:20:22.062862 11491 master_runner.cc:305] Clearing existing system tablet
I20260810 07:20:22.063153 11491 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:22.063215 11491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:22.064759 11491 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:20:22.065802 11643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.066059 11491 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:22.066242 11491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:22.066291 11491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/wal
uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "d6d674b8586e90b0b984f1295f316daa"
server_key_iv: "67afd732e5f6dde71de5b968e8719107"
server_key_version: "encryptionkey@0"
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.125.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.252.keytab.
Entry for principal oryx/127.1.125.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.252.keytab.
I20260810 07:20:22.067975 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.252:37325
--webserver_interface=127.1.125.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.252.keytab
--principal=oryx/127.1.125.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:45317,127.1.125.253:34723,127.1.125.252:37325
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:20:22.068725 11491 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:22.071180 11491 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 07:20:22.071290 11491 log.cc:1196] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 07:20:22.071601 11491 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: Deleting consensus metadata
I20260810 07:20:22.071836 11491 master_runner.cc:315] Copying system tablet from 127.1.125.254:45317
I20260810 07:20:22.073114 11491 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: tablet copy: Beginning tablet copy session from remote peer at address 127.1.125.254:45317
I20260810 07:20:22.081502 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.073801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44888 (local address 127.1.125.254:45317)
0810 07:20:22.073945 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.073950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.074183 (+ 233us) server_negotiation.cc:407] Connection header received
0810 07:20:22.074335 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.074338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.074395 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.074493 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.075224 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.075764 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.076676 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.076913 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.077041 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.078209 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.078213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.078216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.078241 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.079760 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.080218 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.080222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.080224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.080305 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.080660 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.080663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.080665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.080843 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.080956 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.081227 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.081342 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:20:22.081699 11650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.073727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:44888)
0810 07:20:22.074078 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.074090 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.074234 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.074532 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.074535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.074728 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.075091 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.075098 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.075879 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.075885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.076534 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.076548 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.077051 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.077055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.077156 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.077716 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.077737 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.078061 (+ 324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.079889 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.079892 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.079894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.080107 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.080402 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.080405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.080407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.080561 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.080947 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.080950 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.081059 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.081172 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.081328 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":84,"threads_started":1}
I20260810 07:20:22.082211 10831 tablet_copy_service.cc:140] P 94a50c31fbb042299f5f43ac5b7c8755: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e62a6dfd26a74cb7ba4d14bac5f9c360 ({username='oryx', principal='oryx/127.1.125.253@KRBTEST.COM'} at 127.0.0.1:44888)
I20260810 07:20:22.082406 10831 tablet_copy_service.cc:161] P 94a50c31fbb042299f5f43ac5b7c8755: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e62a6dfd26a74cb7ba4d14bac5f9c360 at {username='oryx', principal='oryx/127.1.125.253@KRBTEST.COM'} at 127.0.0.1:44888: session id = e62a6dfd26a74cb7ba4d14bac5f9c360-00000000000000000000000000000000
I20260810 07:20:22.083985 10831 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 07:20:22.085538 11491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 07:20:22.087496 11491 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: tablet copy: Starting download of 0 data blocks...
I20260810 07:20:22.087713 11491 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: tablet copy: Starting download of 1 WAL segments...
I20260810 07:20:22.089438 11491 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 07:20:22.091089 10831 tablet_copy_service.cc:342] P 94a50c31fbb042299f5f43ac5b7c8755: Request end of tablet copy session e62a6dfd26a74cb7ba4d14bac5f9c360-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.125.253@KRBTEST.COM'} at 127.0.0.1:44888
I20260810 07:20:22.091202 10831 tablet_copy_service.cc:433] P 94a50c31fbb042299f5f43ac5b7c8755: ending tablet copy session e62a6dfd26a74cb7ba4d14bac5f9c360-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e62a6dfd26a74cb7ba4d14bac5f9c360
I20260810 07:20:22.098330 11491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:22.100014 11655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.100199 11491 server_base.cc:1034] running on GCE node
W20260810 07:20:22.100347 11652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.100843 11653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.101151 11491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:22.101456 11491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:22.102603 11491 hybrid_clock.cc:648] HybridClock initialized: now 1786346422102588 us; error 30 us; skew 500 ppm
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:22.104184 11491 init.cc:377] Logged in from keytab as oryx/127.1.125.253@KRBTEST.COM (short username oryx)
I20260810 07:20:22.104707 11491 webserver.cc:460] Webserver started at http://127.1.125.253:46881/ using document root <none> and password file <none>
I20260810 07:20:22.104908 11491 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:22.104965 11491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:22.106074 11491 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:22.107151 11666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.107401 11491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:22.107506 11491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/wal
uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "d6d674b8586e90b0b984f1295f316daa"
server_key_iv: "67afd732e5f6dde71de5b968e8719107"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.107612 11491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:22.121670 11491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:22.123829 11491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:22.129397 11491 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.253:34723
I20260810 07:20:22.130050 11491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-1/data/info.pb
I20260810 07:20:22.131552 11752 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:20:22.132390 11752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: Bootstrap starting.
I20260810 07:20:22.140645 11751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.253:34723 every 8 connection(s)
I20260810 07:20:22.140825 11752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:20:22.141196 11752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: Bootstrap complete.
I20260810 07:20:22.141727 11752 raft_consensus.cc:348] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true } }
I20260810 07:20:22.141870 11752 raft_consensus.cc:729] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e62a6dfd26a74cb7ba4d14bac5f9c360, State: Initialized, Role: LEARNER
I20260810 07:20:22.141992 11752 consensus_queue.cc:260] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true } }
I20260810 07:20:22.142340 11756 sys_catalog.cc:455] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true } } }
I20260810 07:20:22.142505 11756 sys_catalog.cc:458] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: This master's current role is: LEARNER
I20260810 07:20:22.142522 11752 sys_catalog.cc:564] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:22.145960 11770 catalog_manager.cc:1259] Loaded cluster ID: da4a4ebf438b41e1bb371235fd054f0f
I20260810 07:20:22.146127 11770 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: loading cluster ID for follower catalog manager: success
I20260810 07:20:22.148267 11770 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: acquiring CA information for follower catalog manager: success
I20260810 07:20:22.149534 11770 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 07:20:22.240969 11644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:22.241279 11644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:22.241380 11644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:22.246024 11644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:22.246112 11644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:22.246151 11644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:22.246188 11644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:22.246222 11644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:22.251820 11644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:45317,127.1.125.253:34723,127.1.125.252:37325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.252.keytab
--principal=oryx/127.1.125.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.252:37325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.125.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:22.253111 11644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:22.254141 11644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:22.263396 11781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.263449 11778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.263495 11776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.263726 11644 server_base.cc:1034] running on GCE node
I20260810 07:20:22.264400 11644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:22.265575 11644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:22.266775 11644 hybrid_clock.cc:648] HybridClock initialized: now 1786346422266760 us; error 28 us; skew 500 ppm
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:22.269778 11644 init.cc:377] Logged in from keytab as oryx/127.1.125.252@KRBTEST.COM (short username oryx)
I20260810 07:20:22.270619 11644 webserver.cc:460] Webserver started at http://127.1.125.252:37283/ using document root <none> and password file <none>
I20260810 07:20:22.271139 11644 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:22.271196 11644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:22.271366 11644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:22.273533 11644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/data/instance:
uuid: "a5b3b5cc29de475c85bdd203ee9023f5"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-44d2"
server_key: "1dd8c90cf666823d9a68c7193d80d85c"
server_key_iv: "223463dba86efab88c74963e64993bdb"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.274052 11644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/wal/instance:
uuid: "a5b3b5cc29de475c85bdd203ee9023f5"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-44d2"
server_key: "1dd8c90cf666823d9a68c7193d80d85c"
server_key_iv: "223463dba86efab88c74963e64993bdb"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.275490 11077 heartbeater.cc:503] Master 127.1.125.254:45317 was elected leader, sending a full tablet report...
I20260810 07:20:22.278020 11644 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:22.280397 11789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.281764 11644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:22.281919 11644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/wal
uuid: "a5b3b5cc29de475c85bdd203ee9023f5"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-44d2"
server_key: "1dd8c90cf666823d9a68c7193d80d85c"
server_key_iv: "223463dba86efab88c74963e64993bdb"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.282027 11644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:22.293393 11644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:22.296919 11644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:22.297194 11644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:22.305528 11644 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.252:37325
I20260810 07:20:22.305828 11866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.252:37325 every 8 connection(s)
I20260810 07:20:22.306571 11644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/data/info.pb
I20260810 07:20:22.307493 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 11644
I20260810 07:20:22.310451 11868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:22.315436 11868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.125.252@KRBTEST.COM
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.252@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:22.324057 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.309204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41632 (local address 127.1.125.252:37325)
0810 07:20:22.309644 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.309665 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.309711 (+ 46us) server_negotiation.cc:407] Connection header received
0810 07:20:22.310769 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.310792 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.311228 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.311599 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.312400 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.314025 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.314881 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.315569 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.315735 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.318670 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.318677 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.318691 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.318723 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.320680 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.321121 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.321129 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.321137 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.321268 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.321729 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.321736 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.321739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.322228 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.322479 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.322826 (+ 347us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.323172 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":108,"threads_started":1}
I20260810 07:20:22.327783 11873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.317406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:44904)
0810 07:20:22.317865 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.317905 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.318059 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.318408 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.318413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.318939 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.319105 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.319112 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.320065 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.320069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.321017 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.321026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.321531 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.321535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.321669 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.322213 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.322225 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.323678 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.325793 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.325801 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.325805 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.326115 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.326475 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.326478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.326480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.326579 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.327131 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.327136 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.327300 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.327421 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.327568 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"spinlock_wait_cycles":7424,"thread_start_us":207,"threads_started":1}
I20260810 07:20:22.327762 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.317541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44904 (local address 127.1.125.254:45317)
0810 07:20:22.317692 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.317696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.317988 (+ 292us) server_negotiation.cc:407] Connection header received
0810 07:20:22.318190 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.318196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.318256 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.318351 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.319274 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.319938 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.321167 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.321383 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.321516 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.323822 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.323826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.323828 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.323861 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.325678 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.326235 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.326238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.326240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.326334 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.326808 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.326812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.326814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.327007 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.327086 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.327521 (+ 435us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.327640 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:20:22.329149 11868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.252@KRBTEST.COM for oryx/127.1.125.253@KRBTEST.COM
I20260810 07:20:22.337136 11877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.329779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46130 (local address 127.1.125.253:34723)
0810 07:20:22.330051 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.330054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.330066 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:22.330108 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.330110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.330149 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.330223 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.330578 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.331224 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.331953 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.332179 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.332254 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.334077 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.334080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.334082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.334104 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.335621 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.336025 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.336027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.336028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.336101 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.336329 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.336331 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.336332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.336576 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.336657 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.336908 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.337012 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":120,"threads_started":1}
I20260810 07:20:22.337154 11873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.329648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:34723 (local address 127.0.0.1:46130)
0810 07:20:22.329771 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.329786 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.329856 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.330251 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.330253 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.330384 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.330475 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.330479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.331326 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.331329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.331847 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.331855 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.332271 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.332274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.332350 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.332772 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.332784 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.333945 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.335726 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.335735 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.335736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.335939 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.336180 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.336182 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.336183 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.336253 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.336674 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.336677 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.336757 (+ 80us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.336850 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.337000 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:20:22.337867 11868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.252@KRBTEST.COM for oryx/127.1.125.252@KRBTEST.COM
I20260810 07:20:22.345645 11873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.338314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:37325 (local address 127.0.0.1:41648)
0810 07:20:22.338428 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.338444 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.338529 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.338708 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.338710 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.338830 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.338896 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.338900 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.339933 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.339936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.340420 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.340427 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.340920 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.340923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.340985 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.341429 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.341442 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.342637 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.344325 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.344327 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.344328 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.344640 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.344897 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.344899 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.344900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.344973 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.345265 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.345267 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.345336 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.345413 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.345517 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 07:20:22.345683 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.338404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.1.125.252:37325)
0810 07:20:22.338521 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.338524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.338536 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:22.338602 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.338604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.338638 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.338694 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.338986 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.339456 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.340608 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.340824 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.340903 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.342790 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.342795 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.342797 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.342823 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.344226 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.344724 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.344727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.344728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.344806 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.345032 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.345034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.345035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.345187 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.345258 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.345480 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.345569 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:20:22.348841 11868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: Bootstrap starting.
I20260810 07:20:22.351112 11868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:22.351852 11868 log.cc:824] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:22.353919 11868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: No bootstrap required, opened a new log
I20260810 07:20:22.357043 11868 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } }
I20260810 07:20:22.357239 11868 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:22.357275 11868 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5b3b5cc29de475c85bdd203ee9023f5, State: Initialized, Role: FOLLOWER
I20260810 07:20:22.357645 11868 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } }
I20260810 07:20:22.358229 11879 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } } }
I20260810 07:20:22.358366 11879 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:20:22.358978 11868 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 07:20:22.364593 11895 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:20:22.364717 11895 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:20:22.369521 11644 master_runner.cc:428] Detected that this master a5b3b5cc29de475c85bdd203ee9023f5 is joining an existing cluster
I20260810 07:20:22.369602 11644 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.125.252:37325
I20260810 07:20:22.377681 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 10896
I20260810 07:20:22.386515 11907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.375648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:34723 (local address 127.0.0.1:46142)
0810 07:20:22.375942 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.375955 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.376032 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.377077 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.377080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.377362 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.377524 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.377533 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.378681 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.378684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.379716 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.379734 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.380520 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.380524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.380625 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.381251 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.381266 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.381746 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.384283 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.384287 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.384289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.384716 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.385069 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.385072 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.385074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.385161 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.385933 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.385935 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.386062 (+ 127us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.386170 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.386310 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":51,"thread_start_us":99,"threads_started":1}
I20260810 07:20:22.387287 11877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.376398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.1.125.253:34723)
0810 07:20:22.376696 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.376701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.376715 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:20:22.376770 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.376773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.376824 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.376927 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.377672 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.378529 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.379874 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.380227 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.380310 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.381917 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.381921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.381923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.381951 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.384152 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.384835 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.384838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.384840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.384955 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.385264 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.385268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.385269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.385762 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.385841 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.386292 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.386491 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206}
I20260810 07:20:22.390808 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.375365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44910 (local address 127.1.125.254:45317)
0810 07:20:22.375540 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.375548 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.375723 (+ 175us) server_negotiation.cc:407] Connection header received
0810 07:20:22.375890 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.375893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.375951 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.376015 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.377727 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.378762 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.381145 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.381640 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.381741 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.385291 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.385299 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.385306 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.385348 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.388298 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.389079 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.389086 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.389091 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.389212 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.389740 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.389744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.389746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.389987 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.390088 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.390515 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.390650 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:20:22.391213 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.375364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:44910)
0810 07:20:22.375640 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.375656 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.375765 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.376174 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.376178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.376472 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.377582 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.377594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.379026 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.379032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.380959 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.380975 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.381977 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.381984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.382126 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.383104 (+ 978us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.383127 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.385084 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.388449 (+ 3365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.388456 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.388461 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.388882 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.389366 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.389432 (+ 66us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.389437 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.389609 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.390107 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.390114 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.390279 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.390443 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.390647 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":4,"thread_start_us":107,"threads_started":1}
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346420, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.254@KRBTEST.COM for oryx/127.1.125.252@KRBTEST.COM
I20260810 07:20:22.413893 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:33107
--local_ip_for_outbound_sockets=127.1.125.193
--tserver_master_addrs=127.1.125.254:45317
--webserver_port=33279
--webserver_interface=127.1.125.193
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab
--principal=oryx/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.125.254:45317,127.1.125.253:34723
--tserver_master_addrs=127.1.125.254:45317,127.1.125.253:34723,127.1.125.252:37325 with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:20:22.426144 11629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.395961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:37325 (local address 127.0.0.1:41678)
0810 07:20:22.396103 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.396121 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.396214 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.396948 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.396952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.397155 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.397266 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.397272 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.398770 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.398774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.399661 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.399672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.400555 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.400559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.400662 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.401226 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.401241 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.403255 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.423972 (+ 20717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.423978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.423981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.424295 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.424740 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.424746 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.424749 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.424861 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.425439 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.425445 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.425576 (+ 131us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.425695 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.425891 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:20:22.426937 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.396009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41678 (local address 127.1.125.252:37325)
0810 07:20:22.396461 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.396474 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.396551 (+ 77us) server_negotiation.cc:407] Connection header received
0810 07:20:22.396642 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.396648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.396737 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.396917 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.397405 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.398624 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.399801 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.400384 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.400574 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.403417 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.403422 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.403425 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.403451 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.423801 (+ 20350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.424466 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.424472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.424500 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.424601 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.425000 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.425006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.425010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.425290 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.426544 (+ 1254us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.426623 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.426748 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":201,"threads_started":1}
I20260810 07:20:22.428673 10804 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.125.252:37325: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5"
member_type: NON_VOTER
last_known_addr {
host: "127.1.125.252"
port: 37325
}
attrs {
promote: true
}
}
dest_uuid: "94a50c31fbb042299f5f43ac5b7c8755"
cas_config_opid_index: 5
I20260810 07:20:22.429066 10804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } }
I20260810 07:20:22.430176 11634 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a5b3b5cc29de475c85bdd203ee9023f5 (127.1.125.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } } }
I20260810 07:20:22.430737 11833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:22.431775 11634 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.125.252:37325
I20260810 07:20:22.431939 11637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } } }
I20260810 07:20:22.432050 11637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:22.432260 11644 master.cc:537] Master@127.1.125.252:37325 shutting down...
I20260810 07:20:22.432294 11833 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 FOLLOWER]: Refusing update from remote peer 94a50c31fbb042299f5f43ac5b7c8755: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 07:20:22.432942 11629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.425842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:34723 (local address 127.0.0.1:46152)
0810 07:20:22.426326 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.426341 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.426431 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.427892 (+ 1461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.427895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.427917 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:22.428009 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.428016 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.429345 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.429349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.430559 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.430570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.432328 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.432332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.432454 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.432641 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.432725 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.432799 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":116,"threads_started":1}
I20260810 07:20:22.433387 11877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.426268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46152 (local address 127.1.125.253:34723)
0810 07:20:22.426517 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.426522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.426536 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:20:22.426590 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.426593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.426660 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.426761 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:22.428169 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.429070 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.430769 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.431458 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.431564 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.431735 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.433175 (+ 1440us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.433267 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260810 07:20:22.433041 11633 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:22.434659 11709 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 LEARNER]: Refusing update from remote peer 94a50c31fbb042299f5f43ac5b7c8755: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 07:20:22.434777 11710 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260810 07:20:22.434939 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.376554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41662 (local address 127.1.125.252:37325)
0810 07:20:22.376650 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.376654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.376667 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:20:22.376700 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.376704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.376740 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.376813 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.406598 (+ 29785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.407378 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.408667 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.409000 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.409074 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.422062 (+ 12988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.422066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.422069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.422091 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.424075 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.433791 (+ 9716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.433795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.433796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.433896 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.434149 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.434152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.434154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.434338 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.434418 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.434675 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.434787 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:20:22.435374 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.376046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:37325 (local address 127.0.0.1:41662)
0810 07:20:22.376269 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.376282 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.376376 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.377030 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.377033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.377312 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.406442 (+ 29130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.406455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.407502 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.407505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.408542 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.408554 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.409209 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.409212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.409312 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.421339 (+ 12027us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.421356 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.421911 (+ 555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.424186 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.424189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.424191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.433694 (+ 9503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.433990 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.433992 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.433994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.434072 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.434460 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.434462 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.434536 (+ 74us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.435044 (+ 508us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.435173 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":4,"thread_start_us":78,"threads_started":1}
I20260810 07:20:22.435330 11756 sys_catalog.cc:455] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94a50c31fbb042299f5f43ac5b7c8755. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true } } }
I20260810 07:20:22.435501 11756 sys_catalog.cc:458] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: This master's current role is: LEARNER
I20260810 07:20:22.435839 11637 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260810 07:20:22.436642 10782 consensus_peers.cc:487] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 -> Peer a5b3b5cc29de475c85bdd203ee9023f5 (127.1.125.252:37325): Couldn't send request to peer a5b3b5cc29de475c85bdd203ee9023f5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:20:22.438012 11710 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a5b3b5cc29de475c85bdd203ee9023f5 (127.1.125.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } } }
I20260810 07:20:22.440008 11756 sys_catalog.cc:455] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: NON_VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: true } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } } }
I20260810 07:20:22.440156 11756 sys_catalog.cc:458] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: This master's current role is: LEARNER
I20260810 07:20:22.440532 11637 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755: attempting to promote NON_VOTER e62a6dfd26a74cb7ba4d14bac5f9c360 to VOTER
I20260810 07:20:22.440971 11637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } }
I20260810 07:20:22.441500 11644 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:22.441777 11644 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:22.441866 11644 tablet_replica.cc:330] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: stopping tablet replica
I20260810 07:20:22.442566 11710 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 LEARNER]: Refusing update from remote peer 94a50c31fbb042299f5f43ac5b7c8755: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260810 07:20:22.442842 10782 consensus_peers.cc:487] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 -> Peer a5b3b5cc29de475c85bdd203ee9023f5 (127.1.125.252:37325): Couldn't send request to peer a5b3b5cc29de475c85bdd203ee9023f5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:20:22.443054 11634 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 07:20:22.447528 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.447159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:37325 (local address 127.0.0.1:41688)
0810 07:20:22.447390 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.447459 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:37325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 07:20:22.453418 11633 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e62a6dfd26a74cb7ba4d14bac5f9c360 (127.1.125.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } } }
I20260810 07:20:22.454906 11633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } } }
I20260810 07:20:22.455014 11633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:22.455176 11633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } } }
I20260810 07:20:22.455250 11633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:22.455534 11710 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e62a6dfd26a74cb7ba4d14bac5f9c360 (127.1.125.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } } }
I20260810 07:20:22.457382 11756 sys_catalog.cc:455] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } } }
I20260810 07:20:22.457508 11756 sys_catalog.cc:458] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:20:22.458384 11644 master.cc:552] Master@127.1.125.252:37325 shutdown complete.
I20260810 07:20:22.458438 11644 master_runner.cc:305] Clearing existing system tablet
I20260810 07:20:22.458691 11644 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:22.458750 11644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:22.460249 11644 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:22.461277 11917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.461530 11644 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:22.461661 11644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:20:22.461706 11644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/wal
uuid: "a5b3b5cc29de475c85bdd203ee9023f5"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-44d2"
server_key: "1dd8c90cf666823d9a68c7193d80d85c"
server_key_iv: "223463dba86efab88c74963e64993bdb"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.463909 11644 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:22.466429 11644 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 07:20:22.466529 11644 log.cc:1196] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 07:20:22.466799 11644 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: Deleting consensus metadata
I20260810 07:20:22.466981 11644 master_runner.cc:315] Copying system tablet from 127.1.125.254:45317
I20260810 07:20:22.468184 11644 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: tablet copy: Beginning tablet copy session from remote peer at address 127.1.125.254:45317
I20260810 07:20:22.478376 11929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.468766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:44920)
0810 07:20:22.469054 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.469071 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.469215 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.469567 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.469570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.469834 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.470028 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.470036 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.471189 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.471193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.472183 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.472196 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.472838 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.472842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.472954 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.474017 (+ 1063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.474032 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.474463 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.476585 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.476588 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.476591 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.476862 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.477285 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.477289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.477291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.477386 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.477855 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.477858 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.477961 (+ 103us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.478061 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.478206 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":100,"threads_started":1}
I20260810 07:20:22.479039 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.469160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44920 (local address 127.1.125.254:45317)
0810 07:20:22.469306 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.469311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.469326 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:20:22.469374 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.469377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.469440 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.469531 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.470165 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.471053 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.472327 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.472709 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.472836 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.474643 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.474647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.474650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.474679 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.476454 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.476971 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.476975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.476977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.477169 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.477503 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.477507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.477509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.477719 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.477810 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.478763 (+ 953us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.478889 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:20:22.480647 10831 tablet_copy_service.cc:140] P 94a50c31fbb042299f5f43ac5b7c8755: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a5b3b5cc29de475c85bdd203ee9023f5 ({username='oryx', principal='oryx/127.1.125.252@KRBTEST.COM'} at 127.0.0.1:44920)
I20260810 07:20:22.480762 10831 tablet_copy_service.cc:161] P 94a50c31fbb042299f5f43ac5b7c8755: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a5b3b5cc29de475c85bdd203ee9023f5 at {username='oryx', principal='oryx/127.1.125.252@KRBTEST.COM'} at 127.0.0.1:44920: session id = a5b3b5cc29de475c85bdd203ee9023f5-00000000000000000000000000000000
I20260810 07:20:22.482308 10831 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 07:20:22.483727 11644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 07:20:22.486403 11644 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: tablet copy: Starting download of 0 data blocks...
I20260810 07:20:22.486610 11644 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: tablet copy: Starting download of 1 WAL segments...
I20260810 07:20:22.488227 11644 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 07:20:22.490180 10831 tablet_copy_service.cc:342] P 94a50c31fbb042299f5f43ac5b7c8755: Request end of tablet copy session a5b3b5cc29de475c85bdd203ee9023f5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.125.252@KRBTEST.COM'} at 127.0.0.1:44920
I20260810 07:20:22.490265 10831 tablet_copy_service.cc:433] P 94a50c31fbb042299f5f43ac5b7c8755: ending tablet copy session a5b3b5cc29de475c85bdd203ee9023f5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a5b3b5cc29de475c85bdd203ee9023f5
I20260810 07:20:22.497741 11644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:22.499492 11931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.499902 11939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.499915 11289 heartbeater.cc:503] Master 127.1.125.254:45317 was elected leader, sending a full tablet report...
W20260810 07:20:22.499536 11932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.500655 11644 server_base.cc:1034] running on GCE node
I20260810 07:20:22.500831 11644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:22.501149 11644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:22.502293 11644 hybrid_clock.cc:648] HybridClock initialized: now 1786346422502279 us; error 29 us; skew 500 ppm
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:22.504032 11644 init.cc:377] Logged in from keytab as oryx/127.1.125.252@KRBTEST.COM (short username oryx)
I20260810 07:20:22.504549 11644 webserver.cc:460] Webserver started at http://127.1.125.252:34931/ using document root <none> and password file <none>
I20260810 07:20:22.504778 11644 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:22.504832 11644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:22.506439 11644 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:20:22.507539 11952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.507848 11644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:20:22.507946 11644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/wal
uuid: "a5b3b5cc29de475c85bdd203ee9023f5"
format_stamp: "Formatted at 2026-08-10 07:20:22 on dist-test-slave-44d2"
server_key: "1dd8c90cf666823d9a68c7193d80d85c"
server_key_iv: "223463dba86efab88c74963e64993bdb"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.508039 11644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:22.514315 11644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:22.516592 11644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:22.521931 11644 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.252:37325
I20260810 07:20:22.521956 12027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.252:37325 every 8 connection(s)
I20260810 07:20:22.522549 11644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/master-2/data/info.pb
I20260810 07:20:22.524398 12028 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:20:22.525610 12028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: Bootstrap starting.
I20260810 07:20:22.538746 12028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:20:22.539093 12028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: Bootstrap complete.
I20260810 07:20:22.539801 12028 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } }
I20260810 07:20:22.539966 12028 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a5b3b5cc29de475c85bdd203ee9023f5, State: Initialized, Role: LEARNER
I20260810 07:20:22.540082 12028 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } }
I20260810 07:20:22.540530 12031 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } } }
I20260810 07:20:22.540606 12028 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:22.540726 12031 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: This master's current role is: LEARNER
I20260810 07:20:22.545490 12055 catalog_manager.cc:1259] Loaded cluster ID: da4a4ebf438b41e1bb371235fd054f0f
I20260810 07:20:22.545562 12055 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: loading cluster ID for follower catalog manager: success
I20260810 07:20:22.547996 12055 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: acquiring CA information for follower catalog manager: success
I20260810 07:20:22.549516 12055 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 07:20:22.571810 11912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:22.572095 11912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:22.572175 11912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:22.576779 11912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:22.576858 11912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:22.576963 11912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:22.582983 11912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab
--principal=oryx/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:33107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=33279
--tserver_master_addrs=127.1.125.254:45317,127.1.125.253:34723,127.1.125.252:37325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:22.584082 11912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:22.585212 11912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:22.595553 12064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.595629 12060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.595675 11912 server_base.cc:1034] running on GCE node
W20260810 07:20:22.595583 12061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.596215 11912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:22.596916 11912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:22.598117 11912 hybrid_clock.cc:648] HybridClock initialized: now 1786346422598095 us; error 31 us; skew 500 ppm
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:22.601126 11912 init.cc:377] Logged in from keytab as oryx/127.1.125.193@KRBTEST.COM (short username oryx)
I20260810 07:20:22.602102 11912 webserver.cc:460] Webserver started at http://127.1.125.193:33279/ using document root <none> and password file <none>
I20260810 07:20:22.602699 11912 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:22.602803 11912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:22.606931 11912 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:20:22.609411 12072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.610610 11912 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:22.610754 11912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "06f27bfb4699425ead7908ac41157923"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "7ff91d5028c8fa3fb914921ab423ab82"
server_key_iv: "bcf76f7a5eef65444ae324683bd6b370"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.611378 11912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:22.630074 11912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:22.633055 11912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:22.633282 11912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:22.633911 11912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:22.634815 11912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:22.634889 11912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.634965 11912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:22.635010 11912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.648419 11912 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:33107
I20260810 07:20:22.648437 12251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:33107 every 8 connection(s)
I20260810 07:20:22.649919 11912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:22.656526 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 11912
I20260810 07:20:22.656772 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 11082
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.252@KRBTEST.COM
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.253@KRBTEST.COM
I20260810 07:20:22.674441 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.652127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:40931 (local address 127.1.125.252:37325)
0810 07:20:22.654765 (+ 2638us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.654770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.654786 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:20:22.654838 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.654842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.654902 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.655014 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.656775 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.658023 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.659648 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.660001 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.660095 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.666791 (+ 6696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.666799 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.666804 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.666838 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.668550 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.669018 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.669023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.669027 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.669179 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.670024 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.670029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.670034 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.670282 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.670402 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.673907 (+ 3505us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.674131 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2549,"thread_start_us":443,"threads_started":1}
I20260810 07:20:22.676271 11877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.654157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:48583 (local address 127.1.125.253:34723)
0810 07:20:22.654278 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.654282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.654298 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:20:22.654960 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.654963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.655016 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.655087 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.656785 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.657818 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.659901 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.660405 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.660540 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.664291 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.664297 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.664303 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.664336 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.666032 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.669323 (+ 3291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.669330 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.669335 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.669431 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.674767 (+ 5336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.674775 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.674780 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.675155 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.675289 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.675822 (+ 533us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.676010 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:20:22.677085 12260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.653781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:34723 (local address 127.1.125.193:48583)
0810 07:20:22.654029 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.654043 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.654829 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.655272 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.655279 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.655922 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.656577 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.656604 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.658051 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.658057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.659729 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.659745 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.660746 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.660752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.660899 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.661387 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.661410 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.664152 (+ 2742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.666178 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.666186 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.666204 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.666543 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.669562 (+ 3019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.669567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.669571 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.669772 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.675379 (+ 5607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.675385 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.675546 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.676454 (+ 908us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.676723 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":129,"spinlock_wait_cycles":2304,"thread_start_us":87,"threads_started":1}
I20260810 07:20:22.677582 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.652614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:37325 (local address 127.1.125.193:40931)
0810 07:20:22.653199 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.653254 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.654538 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.655270 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.655277 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.655914 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.656576 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.656617 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.658144 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.658149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.659513 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.659527 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.660260 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.660266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.660516 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.661364 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.661387 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.663779 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.668683 (+ 4904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.668688 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.668693 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.668921 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.669849 (+ 928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.669854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.669858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.669929 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.670498 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.670511 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.671031 (+ 520us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.671365 (+ 334us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.671925 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":154,"spinlock_wait_cycles":18304,"thread_start_us":153,"threads_started":1}
I20260810 07:20:22.684198 12253 heartbeater.cc:347] Connected to a master server at 127.1.125.252:37325
I20260810 07:20:22.684284 12255 heartbeater.cc:347] Connected to a master server at 127.1.125.253:34723
I20260810 07:20:22.684445 12253 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:22.684445 12255 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:22.685222 12255 heartbeater.cc:511] Master 127.1.125.253:34723 requested a full tablet report, sending...
I20260810 07:20:22.685225 12253 heartbeater.cc:511] Master 127.1.125.252:37325 requested a full tablet report, sending...
I20260810 07:20:22.687247 11680 ts_manager.cc:194] Registered new tserver with Master: 06f27bfb4699425ead7908ac41157923 (127.1.125.193:33107)
I20260810 07:20:22.687247 11968 ts_manager.cc:194] Registered new tserver with Master: 06f27bfb4699425ead7908ac41157923 (127.1.125.193:33107)
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:22.690840 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:33527
--local_ip_for_outbound_sockets=127.1.125.194
--tserver_master_addrs=127.1.125.254:45317
--webserver_port=35477
--webserver_interface=127.1.125.194
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab
--principal=oryx/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.125.254:45317,127.1.125.253:34723
--tserver_master_addrs=127.1.125.254:45317,127.1.125.253:34723,127.1.125.252:37325 with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:20:22.693192 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.652643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.193:56245)
0810 07:20:22.653729 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.653743 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.654538 (+ 795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.655272 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.655279 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.655922 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.682166 (+ 26244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.682183 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.683539 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.683546 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.684653 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.684670 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.685602 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.685608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.685747 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.686538 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.686558 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.689362 (+ 2804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.691516 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.691519 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.691521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.691720 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.692047 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.692049 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.692051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.692128 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.692719 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.692724 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.692827 (+ 103us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.692909 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.693031 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1014,"mutex_wait_us":25864,"spinlock_wait_cycles":13952,"thread_start_us":143,"threads_started":1}
I20260810 07:20:22.693398 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.652109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:56245 (local address 127.1.125.254:45317)
0810 07:20:22.652451 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.652458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.653828 (+ 1370us) server_negotiation.cc:407] Connection header received
0810 07:20:22.654801 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.654804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.654878 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.654936 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.682420 (+ 27484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.683354 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.684821 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.685262 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.685366 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.689535 (+ 4169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.689538 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.689540 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.689571 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.691380 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.691832 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.691839 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.691843 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.691948 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.692229 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.692235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.692239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.692530 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.692660 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.693034 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.693185 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230}
I20260810 07:20:22.694155 12254 heartbeater.cc:347] Connected to a master server at 127.1.125.254:45317
I20260810 07:20:22.694279 12254 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:22.694482 12254 heartbeater.cc:511] Master 127.1.125.254:45317 requested a full tablet report, sending...
I20260810 07:20:22.695348 10804 ts_manager.cc:194] Re-registered known tserver with Master: 06f27bfb4699425ead7908ac41157923 (127.1.125.193:33107)
I20260810 07:20:22.695974 10804 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:56245
I20260810 07:20:22.738487 11486 heartbeater.cc:503] Master 127.1.125.254:45317 was elected leader, sending a full tablet report...
W20260810 07:20:22.812789 12261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:22.813047 12261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:22.813122 12261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:22.817443 12261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:22.817500 12261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:22.817592 12261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:22.822789 12261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab
--principal=oryx/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:33527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=35477
--tserver_master_addrs=127.1.125.254:45317,127.1.125.253:34723,127.1.125.252:37325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:22.823882 12261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:22.825023 12261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:22.835386 12274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:22.835394 12267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.835498 12261 server_base.cc:1034] running on GCE node
W20260810 07:20:22.835394 12271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:22.836030 12261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:22.836755 12261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:22.838018 12261 hybrid_clock.cc:648] HybridClock initialized: now 1786346422838002 us; error 27 us; skew 500 ppm
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:22.841123 12261 init.cc:377] Logged in from keytab as oryx/127.1.125.194@KRBTEST.COM (short username oryx)
I20260810 07:20:22.842159 12261 webserver.cc:460] Webserver started at http://127.1.125.194:35477/ using document root <none> and password file <none>
I20260810 07:20:22.842756 12261 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:22.842875 12261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:22.847124 12261 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:20:22.849313 12283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.850533 12261 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:20:22.850657 12261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "ce643b0a17074c9e9a974c4ac36f711e"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "b24ee46815ccbef74af851d7f880017d"
server_key_iv: "855cfe9dfd5f435e972c13a2cab5fbef"
server_key_version: "encryptionkey@0"
I20260810 07:20:22.851205 12261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:22.873106 12261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:22.876747 12261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:22.876959 12261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:22.877624 12261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:22.878520 12261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:22.878571 12261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.878612 12261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:22.878666 12261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:22.892256 12261 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:33527
I20260810 07:20:22.892283 12445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:33527 every 8 connection(s)
I20260810 07:20:22.893463 12261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:22.898744 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 12261
I20260810 07:20:22.898989 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 11297
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.253@KRBTEST.COM
I20260810 07:20:22.922148 11877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.897322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:38941 (local address 127.1.125.253:34723)
0810 07:20:22.897468 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.897473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.897546 (+ 73us) server_negotiation.cc:407] Connection header received
0810 07:20:22.898452 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.898456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.898515 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.898599 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.909663 (+ 11064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.910376 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.912112 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.912458 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.912562 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.916777 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.916782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.916784 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.916812 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.919230 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.920191 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.920197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.920201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.920314 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.920757 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.920763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.920767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.921020 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.921096 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.921772 (+ 676us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.921910 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 07:20:22.923780 12455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.897127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:34723 (local address 127.1.125.194:38941)
0810 07:20:22.897473 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.897487 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.897788 (+ 301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.898747 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.898750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.908679 (+ 9929us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.909457 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.909487 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.910548 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.910553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.911961 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.911976 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.912806 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.912812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.913030 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.914416 (+ 1386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.914437 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.916564 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.919384 (+ 2820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.919407 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.919433 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.920060 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.920420 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.920425 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.920429 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.920654 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.921177 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.921185 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.921499 (+ 314us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.922386 (+ 887us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.922756 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":147,"threads_started":1}
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:22.932001 12448 heartbeater.cc:347] Connected to a master server at 127.1.125.253:34723
I20260810 07:20:22.932310 12448 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:22.933167 12448 heartbeater.cc:511] Master 127.1.125.253:34723 requested a full tablet report, sending...
I20260810 07:20:22.934762 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.895684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:48089 (local address 127.1.125.254:45317)
0810 07:20:22.895854 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.895859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.896734 (+ 875us) server_negotiation.cc:407] Connection header received
0810 07:20:22.898018 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.898023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.898100 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.898174 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.914675 (+ 16501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.915498 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.916948 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.917273 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.917335 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.930218 (+ 12883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.930225 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.930232 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.930270 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.932544 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.933109 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.933115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.933120 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.933220 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.933544 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.933550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.933554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.933833 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.933930 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.934341 (+ 411us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.934494 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:20:22.935233 11680 ts_manager.cc:194] Registered new tserver with Master: ce643b0a17074c9e9a974c4ac36f711e (127.1.125.194:33527)
I20260810 07:20:22.935468 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.896047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.194:48089)
0810 07:20:22.896568 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.896630 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.897788 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.898379 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.898390 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.914051 (+ 15661us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.914428 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.914439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.915732 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.915738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.916817 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.916826 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.917469 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.917471 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.917538 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.927984 (+ 10446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.928000 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.929985 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.932717 (+ 2732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.932720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.932722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.932982 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.933336 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.933342 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.933347 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.933442 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.934002 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.934008 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.934127 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.934975 (+ 848us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.935163 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"spinlock_wait_cycles":15744,"thread_start_us":129,"threads_started":1}
I20260810 07:20:22.936110 12447 heartbeater.cc:347] Connected to a master server at 127.1.125.254:45317
I20260810 07:20:22.936231 12447 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:22.936434 12447 heartbeater.cc:511] Master 127.1.125.254:45317 requested a full tablet report, sending...
I20260810 07:20:22.937218 10804 ts_manager.cc:194] Re-registered known tserver with Master: ce643b0a17074c9e9a974c4ac36f711e (127.1.125.194:33527)
I20260810 07:20:22.938045 10804 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:48089
I20260810 07:20:22.938823 11913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.933203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:37325 (local address 127.0.0.1:41696)
0810 07:20:22.933522 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.933538 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.933645 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.934537 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.934544 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.934572 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:22.934693 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.934701 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.936113 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.936117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.937322 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.937333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.938421 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.938426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.938518 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.938541 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.938612 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.938696 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229}
I20260810 07:20:22.938886 12456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.933864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41696 (local address 127.1.125.252:37325)
0810 07:20:22.934170 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.934175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.934194 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:20:22.934262 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.934267 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.934352 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.934830 (+ 478us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:22.935038 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.935962 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.937789 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.938275 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.938431 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.938628 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.938669 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.938748 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":120,"threads_started":1}
I20260810 07:20:22.939538 11983 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 07:20:22.939968 12031 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94a50c31fbb042299f5f43ac5b7c8755. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } } }
I20260810 07:20:22.940137 12031 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: This master's current role is: LEARNER
I20260810 07:20:22.942653 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:44005
--local_ip_for_outbound_sockets=127.1.125.195
--tserver_master_addrs=127.1.125.254:45317
--webserver_port=41509
--webserver_interface=127.1.125.195
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab
--principal=oryx/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.125.254:45317,127.1.125.253:34723
--tserver_master_addrs=127.1.125.254:45317,127.1.125.253:34723,127.1.125.252:37325 with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:20:22 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346422, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.252@KRBTEST.COM
I20260810 07:20:22.951380 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.895609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:36529 (local address 127.1.125.252:37325)
0810 07:20:22.895766 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:22.895771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:22.896737 (+ 966us) server_negotiation.cc:407] Connection header received
0810 07:20:22.898161 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:22.898168 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:22.898229 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:22.898313 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:22.940947 (+ 42634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.941960 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.943760 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.944224 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.944352 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.947694 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:22.947698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:22.947700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:22.947728 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:22.949603 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.950166 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.950168 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.950169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.950243 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:22.950581 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:22.950583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:22.950584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:22.950752 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:22.950826 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:22.951165 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 07:20:22.951269 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 07:20:22.951583 12452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:22.896047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:37325 (local address 127.1.125.194:36529)
0810 07:20:22.896570 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:22.896631 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:22.897772 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:22.898420 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:22.898425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:22.940599 (+ 42174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:22.940745 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.940756 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.942204 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.942208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.943549 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:22.943567 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:22.944643 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:22.944649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:22.944768 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:22.945584 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:22.945606 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:22.947504 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:22.949740 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.949747 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.949751 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.950053 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.950338 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:22.950343 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:22.950347 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:22.950463 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:22.950852 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:22.950858 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:22.950981 (+ 123us) client_negotiation.cc:769] Sending connection context
0810 07:20:22.951102 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 07:20:22.951282 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":29,"spinlock_wait_cycles":35456,"thread_start_us":190,"threads_started":1}
I20260810 07:20:22.952409 12446 heartbeater.cc:347] Connected to a master server at 127.1.125.252:37325
I20260810 07:20:22.952603 12446 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:22.952917 12446 heartbeater.cc:511] Master 127.1.125.252:37325 requested a full tablet report, sending...
I20260810 07:20:22.953745 11968 ts_manager.cc:194] Registered new tserver with Master: ce643b0a17074c9e9a974c4ac36f711e (127.1.125.194:33527)
W20260810 07:20:23.064534 12457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:23.064778 12457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:23.064849 12457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:23.068979 12457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:23.069032 12457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:23.069123 12457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:23.074716 12457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab
--principal=oryx/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:44005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=41509
--tserver_master_addrs=127.1.125.254:45317,127.1.125.253:34723,127.1.125.252:37325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:23.075752 12457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:23.077030 12457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:23.087333 12463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:23.087343 12462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:23.087348 12465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:23.087397 12457 server_base.cc:1034] running on GCE node
I20260810 07:20:23.088038 12457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:23.088829 12457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:23.090013 12457 hybrid_clock.cc:648] HybridClock initialized: now 1786346423089995 us; error 36 us; skew 500 ppm
Aug 10 07:20:23 dist-test-slave-44d2 krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346423, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:23.092880 12457 init.cc:377] Logged in from keytab as oryx/127.1.125.195@KRBTEST.COM (short username oryx)
I20260810 07:20:23.093736 12457 webserver.cc:460] Webserver started at http://127.1.125.195:41509/ using document root <none> and password file <none>
I20260810 07:20:23.094260 12457 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:23.094372 12457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:23.098186 12457 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:20:23.100155 12479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:23.101447 12457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:23.101572 12457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "3f1bb10376c747af8a925456ebb1c4a8"
format_stamp: "Formatted at 2026-08-10 07:20:21 on dist-test-slave-44d2"
server_key: "a91ec6ec77622ebb7a676da5bf845e31"
server_key_iv: "cb917c645f734c5e21aa7c82803cb730"
server_key_version: "encryptionkey@0"
I20260810 07:20:23.102119 12457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:23.118618 12457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:23.121490 12457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:23.121701 12457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:23.122296 12457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:23.123155 12457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:23.123205 12457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:23.123272 12457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:23.123302 12457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:23.134243 12457 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:44005
I20260810 07:20:23.134320 12647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:44005 every 8 connection(s)
I20260810 07:20:23.135279 12457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:23.141075 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 12457
Aug 10 07:20:23 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346423, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.252@KRBTEST.COM
Aug 10 07:20:23 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346423, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
Aug 10 07:20:23 dist-test-slave-44d2 krb5kdc[10738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346423, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.253@KRBTEST.COM
I20260810 07:20:23.156780 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.137240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:55451 (local address 127.1.125.254:45317)
0810 07:20:23.137374 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.137379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.138401 (+ 1022us) server_negotiation.cc:407] Connection header received
0810 07:20:23.139555 (+ 1154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.139558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.139611 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.139721 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:23.141462 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.142295 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.144350 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.144733 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.144813 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.148909 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:23.148912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:23.148915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:23.148952 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:23.154872 (+ 5920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.155420 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.155423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.155425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.155517 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.155838 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.155841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.155842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.156058 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:23.156131 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.156471 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.156620 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53,"spinlock_wait_cycles":2304}
I20260810 07:20:23.157682 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.137686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:34723 (local address 127.1.125.195:32951)
0810 07:20:23.139728 (+ 2042us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:23.139742 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:23.139884 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:23.140263 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:23.140269 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:23.140607 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:23.141317 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.141330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.142494 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.142497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.144413 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.144428 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.145209 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.145212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.145356 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.146401 (+ 1045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:23.146416 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:23.149140 (+ 2724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:23.154021 (+ 4881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:23.154025 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:23.154027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:23.154396 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:23.154764 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:23.154770 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:23.154773 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:23.154973 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:23.155597 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:23.155605 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:23.155990 (+ 385us) client_negotiation.cc:769] Sending connection context
0810 07:20:23.156191 (+ 201us) client_negotiation.cc:240] Negotiation successful
0810 07:20:23.156598 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1961,"mutex_wait_us":419,"thread_start_us":132,"threads_started":1}
I20260810 07:20:23.157985 12657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.137685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.195:55451)
0810 07:20:23.138253 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:23.138306 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:23.139331 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:23.139907 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:23.139914 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:23.140473 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:23.141276 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.141297 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.142491 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.142495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.144203 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.144215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.145020 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.145025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.145243 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.145962 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:23.145991 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:23.148715 (+ 2724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:23.155053 (+ 6338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:23.155056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:23.155060 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:23.155298 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:23.155633 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:23.155635 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:23.155637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:23.155731 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:23.156207 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:23.156210 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:23.156291 (+ 81us) client_negotiation.cc:769] Sending connection context
0810 07:20:23.157011 (+ 720us) client_negotiation.cc:240] Negotiation successful
0810 07:20:23.157146 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":199,"spinlock_wait_cycles":14080,"thread_start_us":144,"threads_started":1}
I20260810 07:20:23.158612 11877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.137865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:32951 (local address 127.1.125.253:34723)
0810 07:20:23.137993 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.137998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.139813 (+ 1815us) server_negotiation.cc:407] Connection header received
0810 07:20:23.140006 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.140010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.140067 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.140192 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:23.141495 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.142271 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.144598 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.144954 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.145042 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.149299 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:23.149303 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:23.149305 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:23.149332 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:23.152817 (+ 3485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.154538 (+ 1721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.154542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.154544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.154638 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.155088 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.155094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.155098 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.155358 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:23.155535 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.158295 (+ 2760us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.158413 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:20:23.159149 12669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.144190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46158 (local address 127.1.125.253:34723)
0810 07:20:23.147132 (+ 2942us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.147136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.147423 (+ 287us) server_negotiation.cc:407] Connection header received
0810 07:20:23.147565 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.147568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.147623 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.147714 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:23.148849 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.149645 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.150800 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.151151 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.151239 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.153006 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:23.153013 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:23.153017 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:23.153049 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:23.155569 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.156140 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.156144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.156145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.156239 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.156640 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.156643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.156645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.157131 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:23.157271 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.158518 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.158941 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2865,"thread_start_us":79,"threads_started":1}
I20260810 07:20:23.159670 12651 heartbeater.cc:347] Connected to a master server at 127.1.125.254:45317
I20260810 07:20:23.159938 12651 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:23.160612 12651 heartbeater.cc:511] Master 127.1.125.254:45317 requested a full tablet report, sending...
I20260810 07:20:23.160912 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.138941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:53985 (local address 127.1.125.252:37325)
0810 07:20:23.139073 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.139087 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.140184 (+ 1097us) server_negotiation.cc:407] Connection header received
0810 07:20:23.140369 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.140373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.140430 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.140544 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:23.141480 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.142310 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.144350 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.144733 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.144813 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.150116 (+ 5303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:23.150121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:23.150123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:23.150153 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:23.153593 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.154659 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.154665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.154668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.154772 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.155160 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.155166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.155168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.155424 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:23.155507 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.160597 (+ 5090us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.160721 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55,"spinlock_wait_cycles":4864}
I20260810 07:20:23.161239 12665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.143368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44930 (local address 127.1.125.254:45317)
0810 07:20:23.144927 (+ 1559us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.144932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.145283 (+ 351us) server_negotiation.cc:407] Connection header received
0810 07:20:23.145329 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.145332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.145378 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.145465 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:23.146645 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.147431 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.148458 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.148811 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.148902 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.152378 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:23.152382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:23.152384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:23.152414 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:23.158425 (+ 6011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.158912 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.158915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.158916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.159023 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.159957 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.159960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.159962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.160190 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:23.160312 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.160930 (+ 618us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.161072 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1483,"spinlock_wait_cycles":4352,"thread_start_us":86,"threads_started":1}
I20260810 07:20:23.161386 12659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.138816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:37325 (local address 127.1.125.195:53985)
0810 07:20:23.140069 (+ 1253us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:23.140079 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:23.140229 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:23.140682 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:23.140685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:23.140863 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:23.141276 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.141297 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.142491 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.142495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.144203 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.144215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.145295 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.145298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.145594 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.146106 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:23.146122 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:23.149945 (+ 3823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:23.153726 (+ 3781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:23.153734 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:23.153753 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:23.154542 (+ 789us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:23.154909 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:23.154913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:23.154917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:23.155019 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:23.155625 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:23.155630 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:23.160349 (+ 4719us) client_negotiation.cc:769] Sending connection context
0810 07:20:23.161065 (+ 716us) client_negotiation.cc:240] Negotiation successful
0810 07:20:23.161201 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1178,"mutex_wait_us":4895,"thread_start_us":94,"threads_started":1}
I20260810 07:20:23.161872 12652 heartbeater.cc:347] Connected to a master server at 127.1.125.253:34723
I20260810 07:20:23.161975 12652 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:23.162153 12652 heartbeater.cc:511] Master 127.1.125.253:34723 requested a full tablet report, sending...
I20260810 07:20:23.162480 10801 ts_manager.cc:194] Re-registered known tserver with Master: 3f1bb10376c747af8a925456ebb1c4a8 (127.1.125.195:44005)
I20260810 07:20:23.162595 12648 heartbeater.cc:347] Connected to a master server at 127.1.125.252:37325
I20260810 07:20:23.162683 12648 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:23.162856 12648 heartbeater.cc:511] Master 127.1.125.252:37325 requested a full tablet report, sending...
I20260810 07:20:23.162879 11680 ts_manager.cc:194] Registered new tserver with Master: 3f1bb10376c747af8a925456ebb1c4a8 (127.1.125.195:44005)
I20260810 07:20:23.163244 10801 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:55451
I20260810 07:20:23.163558 11968 ts_manager.cc:194] Registered new tserver with Master: 3f1bb10376c747af8a925456ebb1c4a8 (127.1.125.195:44005)
I20260810 07:20:23.164366 12456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.145054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41710 (local address 127.1.125.252:37325)
0810 07:20:23.145183 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.145187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.149235 (+ 4048us) server_negotiation.cc:407] Connection header received
0810 07:20:23.149425 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.149432 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.149485 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.149542 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:23.150589 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.151342 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.152625 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.152983 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.153054 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.156946 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:23.156950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:23.156953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:23.156975 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:23.158847 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.160073 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.160088 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.160090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.160183 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.160543 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.160547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.160548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.160770 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:23.160888 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.164016 (+ 3128us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.164183 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:20:23.165985 10801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:20:23.169057 10801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:23.171576 12674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: NON_VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: true } } }
I20260810 07:20:23.171701 12674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:23.176460 12674 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755: attempting to promote NON_VOTER a5b3b5cc29de475c85bdd203ee9023f5 to VOTER
I20260810 07:20:23.176924 12674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } }
I20260810 07:20:23.178459 11983 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 LEARNER]: Refusing update from remote peer 94a50c31fbb042299f5f43ac5b7c8755: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 07:20:23.178833 11633 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260810 07:20:23.179062 11710 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 FOLLOWER]: Refusing update from remote peer 94a50c31fbb042299f5f43ac5b7c8755: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 07:20:23.179659 11633 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260810 07:20:23.181376 12672 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a5b3b5cc29de475c85bdd203ee9023f5 (127.1.125.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } } }
I20260810 07:20:23.181699 11983 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a5b3b5cc29de475c85bdd203ee9023f5 (127.1.125.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } } }
I20260810 07:20:23.182099 11710 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a5b3b5cc29de475c85bdd203ee9023f5 (127.1.125.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } } }
I20260810 07:20:23.183743 12672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } } }
I20260810 07:20:23.183840 12672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:23.183889 12031 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } } }
I20260810 07:20:23.183924 12672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } } }
I20260810 07:20:23.183988 12031 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:20:23.184001 12672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:23.184098 11633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } } }
I20260810 07:20:23.184187 11633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:23.185194 12685 sys_catalog.cc:455] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "94a50c31fbb042299f5f43ac5b7c8755" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } } }
I20260810 07:20:23.185309 12685 sys_catalog.cc:458] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:20:23.192016 11913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.186073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:33527 (local address 127.0.0.1:40058)
0810 07:20:23.186562 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:23.186580 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:23.186683 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:23.187800 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:23.187804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:23.187822 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:23.187928 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.187934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.189543 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.189547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.190657 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.190666 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.191658 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.191662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.191753 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.191780 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 07:20:23.191826 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 07:20:23.191890 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376}
I20260810 07:20:23.192564 12688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.186752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:33107 (local address 127.0.0.1:35832)
0810 07:20:23.187518 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:23.187530 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:23.187604 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:23.188633 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:23.188636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:23.188657 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:23.188772 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.188780 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.190110 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.190114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.191313 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.191323 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.192142 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.192146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.192240 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.192262 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:20:23.192329 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 07:20:23.192399 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":33,"thread_start_us":172,"threads_started":1}
I20260810 07:20:23.192636 12692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.187233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35832 (local address 127.1.125.193:33107)
0810 07:20:23.188149 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.188157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.188180 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:20:23.188256 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.188261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.188458 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.188637 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:23.188902 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.189991 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.191441 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.191993 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.192139 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.192300 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.192358 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.192457 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":805,"thread_start_us":77,"threads_started":1}
I20260810 07:20:23.193262 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.186291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40058 (local address 127.1.125.194:33527)
0810 07:20:23.186657 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.186671 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.186696 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:20:23.186849 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.186857 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.187108 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.187834 (+ 726us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:23.188105 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.189394 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.190795 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.191525 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.192521 (+ 996us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.192689 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.192749 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.192831 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":123,"threads_started":1}
I20260810 07:20:23.194659 12690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.186611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:44005 (local address 127.0.0.1:55004)
0810 07:20:23.187281 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:23.187296 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:23.187386 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:23.189262 (+ 1876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:23.189265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:23.189285 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:23.189385 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.189395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.191077 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.191081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.192376 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.192387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.194265 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.194269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.194356 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.194377 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:20:23.194456 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:20:23.194535 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":22,"thread_start_us":110,"threads_started":1}
I20260810 07:20:23.194713 12691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.187128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55004 (local address 127.1.125.195:44005)
0810 07:20:23.188785 (+ 1657us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.188798 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.188821 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:20:23.188899 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.188904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.189118 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.189276 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:23.189486 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.190956 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.192538 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.194138 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.194265 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.194403 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.194487 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.194577 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1510,"thread_start_us":100,"threads_started":1}
I20260810 07:20:23.195868 12152 tablet_service.cc:1467] Processing CreateTablet for tablet 397beeede3624813b81294b36283f8bd (DEFAULT_TABLE table=test-table [id=f10fc6fea7ab4ca4a6fec241de523b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:23.195869 12354 tablet_service.cc:1467] Processing CreateTablet for tablet 397beeede3624813b81294b36283f8bd (DEFAULT_TABLE table=test-table [id=f10fc6fea7ab4ca4a6fec241de523b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:23.197093 12354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 397beeede3624813b81294b36283f8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:23.197098 12152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 397beeede3624813b81294b36283f8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:23.197013 12547 tablet_service.cc:1467] Processing CreateTablet for tablet 397beeede3624813b81294b36283f8bd (DEFAULT_TABLE table=test-table [id=f10fc6fea7ab4ca4a6fec241de523b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:23.197962 12547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 397beeede3624813b81294b36283f8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:23.204793 12694 tablet_bootstrap.cc:492] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8: Bootstrap starting.
I20260810 07:20:23.206182 12696 tablet_bootstrap.cc:492] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923: Bootstrap starting.
I20260810 07:20:23.207762 12694 tablet_bootstrap.cc:654] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:23.208354 12696 tablet_bootstrap.cc:654] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:23.208606 12694 log.cc:824] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:23.208890 12698 tablet_bootstrap.cc:492] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e: Bootstrap starting.
I20260810 07:20:23.210546 12694 tablet_bootstrap.cc:492] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8: No bootstrap required, opened a new log
I20260810 07:20:23.210753 12694 ts_tablet_manager.cc:1397] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:20:23.210850 12698 tablet_bootstrap.cc:654] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:23.211858 12698 log.cc:824] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:23.213013 12696 log.cc:824] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:23.214633 12698 tablet_bootstrap.cc:492] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e: No bootstrap required, opened a new log
I20260810 07:20:23.214847 12698 ts_tablet_manager.cc:1397] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:20:23.215183 12696 tablet_bootstrap.cc:492] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923: No bootstrap required, opened a new log
I20260810 07:20:23.215372 12696 ts_tablet_manager.cc:1397] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.002s
I20260810 07:20:23.215261 12694 raft_consensus.cc:348] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:23.215538 12694 raft_consensus.cc:374] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:23.215613 12694 raft_consensus.cc:729] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1bb10376c747af8a925456ebb1c4a8, State: Initialized, Role: FOLLOWER
I20260810 07:20:23.216095 12694 consensus_queue.cc:260] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:23.217218 12694 ts_tablet_manager.cc:1428] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 07:20:23.217427 12651 heartbeater.cc:503] Master 127.1.125.254:45317 was elected leader, sending a full tablet report...
I20260810 07:20:23.220533 12698 raft_consensus.cc:348] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:23.220813 12698 raft_consensus.cc:374] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:23.220875 12698 raft_consensus.cc:729] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce643b0a17074c9e9a974c4ac36f711e, State: Initialized, Role: FOLLOWER
I20260810 07:20:23.220891 12696 raft_consensus.cc:348] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:23.221136 12696 raft_consensus.cc:374] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:23.221189 12696 raft_consensus.cc:729] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06f27bfb4699425ead7908ac41157923, State: Initialized, Role: FOLLOWER
I20260810 07:20:23.221379 12698 consensus_queue.cc:260] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:23.221647 12696 consensus_queue.cc:260] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:23.222398 12698 ts_tablet_manager.cc:1428] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 07:20:23.222568 12696 ts_tablet_manager.cc:1428] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 07:20:23.222739 12254 heartbeater.cc:503] Master 127.1.125.254:45317 was elected leader, sending a full tablet report...
I20260810 07:20:23.224315 12447 heartbeater.cc:503] Master 127.1.125.254:45317 was elected leader, sending a full tablet report...
I20260810 07:20:23.328779 12705 raft_consensus.cc:482] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:23.329034 12705 raft_consensus.cc:504] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:23.330272 12705 leader_election.cc:288] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce643b0a17074c9e9a974c4ac36f711e (127.1.125.194:33527), 3f1bb10376c747af8a925456ebb1c4a8 (127.1.125.195:44005)
I20260810 07:20:23.334519 12691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.330775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:57059 (local address 127.1.125.195:44005)
0810 07:20:23.330917 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.330923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.331176 (+ 253us) server_negotiation.cc:407] Connection header received
0810 07:20:23.331298 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.331302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.331352 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.331435 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:23.331744 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.332345 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.333534 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.333929 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.334057 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.334124 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.334303 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.334384 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:20:23.334549 12708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.330553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:33527 (local address 127.1.125.193:33557)
0810 07:20:23.331194 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:23.331210 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:23.331328 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:23.331643 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:23.331645 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:23.331661 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:23.331804 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.331811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.332556 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.332560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.333402 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.333412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.334090 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.334094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.334207 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.334225 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 07:20:23.334326 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 07:20:23.334389 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":96,"thread_start_us":134,"threads_started":1}
I20260810 07:20:23.334558 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.330676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:33557 (local address 127.1.125.194:33527)
0810 07:20:23.330827 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.330832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.331296 (+ 464us) server_negotiation.cc:407] Connection header received
0810 07:20:23.331459 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.331463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.331530 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.331613 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:23.331915 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.332412 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.333545 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.333944 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.334078 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.334130 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.334361 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.334440 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 07:20:23.334554 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.330553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:44005 (local address 127.1.125.193:57059)
0810 07:20:23.330988 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:23.331000 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:23.331188 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:23.331456 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:23.331459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:23.331478 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:23.331625 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.331633 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.332458 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.332463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.333399 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:23.333409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.334063 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.334067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:23.334165 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.334189 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:20:23.334261 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 07:20:23.334348 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":3,"thread_start_us":157,"threads_started":1}
I20260810 07:20:23.335457 12577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "397beeede3624813b81294b36283f8bd" candidate_uuid: "06f27bfb4699425ead7908ac41157923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1bb10376c747af8a925456ebb1c4a8" is_pre_election: true
I20260810 07:20:23.335777 12577 raft_consensus.cc:2454] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06f27bfb4699425ead7908ac41157923 in term 0.
I20260810 07:20:23.336372 12073 leader_election.cc:302] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06f27bfb4699425ead7908ac41157923, 3f1bb10376c747af8a925456ebb1c4a8; no voters:
I20260810 07:20:23.336385 12377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "397beeede3624813b81294b36283f8bd" candidate_uuid: "06f27bfb4699425ead7908ac41157923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce643b0a17074c9e9a974c4ac36f711e" is_pre_election: true
I20260810 07:20:23.336707 12377 raft_consensus.cc:2454] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06f27bfb4699425ead7908ac41157923 in term 0.
I20260810 07:20:23.336727 12705 raft_consensus.cc:2781] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:23.336822 12705 raft_consensus.cc:482] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:23.336884 12705 raft_consensus.cc:3037] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:23.338050 12705 raft_consensus.cc:504] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:23.338429 12705 leader_election.cc:288] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [CANDIDATE]: Term 1 election: Requested vote from peers ce643b0a17074c9e9a974c4ac36f711e (127.1.125.194:33527), 3f1bb10376c747af8a925456ebb1c4a8 (127.1.125.195:44005)
I20260810 07:20:23.338841 12377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "397beeede3624813b81294b36283f8bd" candidate_uuid: "06f27bfb4699425ead7908ac41157923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce643b0a17074c9e9a974c4ac36f711e"
I20260810 07:20:23.338832 12577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "397beeede3624813b81294b36283f8bd" candidate_uuid: "06f27bfb4699425ead7908ac41157923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1bb10376c747af8a925456ebb1c4a8"
I20260810 07:20:23.338975 12577 raft_consensus.cc:3037] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:23.338980 12377 raft_consensus.cc:3037] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:23.340083 12577 raft_consensus.cc:2454] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06f27bfb4699425ead7908ac41157923 in term 1.
I20260810 07:20:23.340286 12377 raft_consensus.cc:2454] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06f27bfb4699425ead7908ac41157923 in term 1.
I20260810 07:20:23.340417 12073 leader_election.cc:302] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06f27bfb4699425ead7908ac41157923, 3f1bb10376c747af8a925456ebb1c4a8; no voters:
I20260810 07:20:23.340652 12705 raft_consensus.cc:2781] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:23.340895 12705 raft_consensus.cc:686] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [term 1 LEADER]: Becoming Leader. State: Replica: 06f27bfb4699425ead7908ac41157923, State: Running, Role: LEADER
I20260810 07:20:23.341266 12705 consensus_queue.cc:237] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:23.345099 10801 catalog_manager.cc:5495] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06f27bfb4699425ead7908ac41157923 (127.1.125.193). New cstate: current_term: 1 leader_uuid: "06f27bfb4699425ead7908ac41157923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } health_report { overall_health: UNKNOWN } } }
I20260810 07:20:23.384196 12692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.381022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35840 (local address 127.1.125.193:33107)
0810 07:20:23.381158 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.381161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.381174 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:20:23.381253 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.381255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.381309 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.381400 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:23.381613 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.382125 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.382903 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.383124 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.383200 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.383472 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:23.383899 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:23.384003 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.384033 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.384081 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 07:20:23.386710 12654 tablet.cc:2369] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:20:23.390682 12577 raft_consensus.cc:1261] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8 [term 1 FOLLOWER]: Refusing update from remote peer 06f27bfb4699425ead7908ac41157923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:23.390683 12377 raft_consensus.cc:1261] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e [term 1 FOLLOWER]: Refusing update from remote peer 06f27bfb4699425ead7908ac41157923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:23.391480 12710 consensus_queue.cc:1035] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:23.391731 12705 consensus_queue.cc:1035] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:20:23.396605 12449 tablet.cc:2369] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:20:23.403330 12719 mvcc.cc:205] Tried to move back new op lower bound from 7316874950200336384 to 7316874950009208832. Current Snapshot: MvccSnapshot[applied={T|T < 7316874950200336384}]
I20260810 07:20:23.413869 10794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:23.414053 10794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:23.418893 10794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94a50c31fbb042299f5f43ac5b7c8755: Sending DeleteTablet for 3 replicas of tablet 397beeede3624813b81294b36283f8bd
I20260810 07:20:23.419648 12354 tablet_service.cc:1514] Processing DeleteTablet for tablet 397beeede3624813b81294b36283f8bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:23 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:40058
I20260810 07:20:23.420177 12730 tablet_replica.cc:330] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e: stopping tablet replica
I20260810 07:20:23.420200 12547 tablet_service.cc:1514] Processing DeleteTablet for tablet 397beeede3624813b81294b36283f8bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:23 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:55004
I20260810 07:20:23.420600 12730 raft_consensus.cc:2229] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:23.420722 12731 tablet_replica.cc:330] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8: stopping tablet replica
I20260810 07:20:23.420981 12730 raft_consensus.cc:2258] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:23.421030 12731 raft_consensus.cc:2229] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:23.421343 12731 raft_consensus.cc:2258] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:23.421464 12152 tablet_service.cc:1514] Processing DeleteTablet for tablet 397beeede3624813b81294b36283f8bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:23 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:35832
I20260810 07:20:23.422855 12731 ts_tablet_manager.cc:1905] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:23.424819 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 10758
I20260810 07:20:23.427971 12730 ts_tablet_manager.cc:1905] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:23.432327 12730 ts_tablet_manager.cc:1918] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:23.432577 12730 log.cc:1196] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/397beeede3624813b81294b36283f8bd
I20260810 07:20:23.433188 12730 ts_tablet_manager.cc:1939] T 397beeede3624813b81294b36283f8bd P ce643b0a17074c9e9a974c4ac36f711e: Deleting consensus metadata
I20260810 07:20:23.436440 12731 ts_tablet_manager.cc:1918] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:23.436793 12732 tablet_replica.cc:330] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923: stopping tablet replica
I20260810 07:20:23.437166 12732 raft_consensus.cc:2229] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:23.437635 12732 raft_consensus.cc:2258] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:23.437835 12731 log.cc:1196] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/397beeede3624813b81294b36283f8bd
I20260810 07:20:23.438261 12731 ts_tablet_manager.cc:1939] T 397beeede3624813b81294b36283f8bd P 3f1bb10376c747af8a925456ebb1c4a8: Deleting consensus metadata
I20260810 07:20:23.442931 12732 ts_tablet_manager.cc:1905] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:23.446864 12732 ts_tablet_manager.cc:1918] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:23.447044 12732 log.cc:1196] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-0/wal/wals/397beeede3624813b81294b36283f8bd
W20260810 07:20:23.447149 12484 connection.cc:535] server connection from 127.0.0.1:55004 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:20:23.447528 12732 ts_tablet_manager.cc:1939] T 397beeede3624813b81294b36283f8bd P 06f27bfb4699425ead7908ac41157923: Deleting consensus metadata
W20260810 07:20:23.447629 12286 connection.cc:535] server connection from 127.0.0.1:40058 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:20:23.448892 12077 connection.cc:441] server connection from 127.0.0.1:35832 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35832 (request call id 1) could send its response
I20260810 07:20:23.450145 12669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.423448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.1.125.253:34723)
0810 07:20:23.423583 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.423588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.423602 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:20:23.423641 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.423645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.423689 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.423768 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:23.437496 (+ 13728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.438428 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.440157 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.440566 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.440681 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.446223 (+ 5542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:23.446228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:23.446234 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:23.446269 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:23.448036 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.448650 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.448654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.448656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.448754 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.449073 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.449089 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.449093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.449318 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:23.449436 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.449819 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.449962 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:20:23.462256 12669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.451897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46174 (local address 127.1.125.253:34723)
0810 07:20:23.452010 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.452014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.452702 (+ 688us) server_negotiation.cc:407] Connection header received
0810 07:20:23.452885 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.452888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.452939 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.452986 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:23.454140 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.454935 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.456251 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.456729 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.456803 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.458473 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:23.458477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:23.458480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:23.458504 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:23.460216 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.460772 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.460779 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.460782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.460870 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.461222 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.461225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.461226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.461455 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:23.461563 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.461935 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.462081 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:20:23.462881 12456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.423579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.1.125.252:37325)
0810 07:20:23.423687 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.423692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.450209 (+ 26517us) server_negotiation.cc:407] Connection header received
0810 07:20:23.450397 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.450401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.450455 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.450508 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:23.452066 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.452941 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.454169 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.454551 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.454628 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.456454 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:23.456458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:23.456461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:23.456512 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:23.459672 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.461248 (+ 1576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.461252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.461254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.461346 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.461653 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.461656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.461658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.462183 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:23.462310 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.462594 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.462729 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:20:23.463083 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:23.452310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41726 (local address 127.1.125.252:37325)
0810 07:20:23.452436 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:23.452442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:23.452702 (+ 260us) server_negotiation.cc:407] Connection header received
0810 07:20:23.452951 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:23.452955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:23.453004 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:23.453069 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:23.454140 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.454918 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.456251 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:23.456595 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:23.456670 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:23.458424 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:23.458431 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:23.458438 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:23.458465 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:23.460611 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.461199 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.461205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.461206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.461299 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:23.461640 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:23.461643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:23.461645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:23.461891 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:23.461988 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:23.462714 (+ 726us) server_negotiation.cc:299] Negotiation successful
0810 07:20:23.462836 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:20:24.404842 12744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.404242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.194:49141)
0810 07:20:24.404615 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.404723 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":83,"threads_started":1}
W20260810 07:20:24.405455 12447 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:45317 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
I20260810 07:20:24.406188 12745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.405477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.195:60073)
0810 07:20:24.405964 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.406059 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":98,"threads_started":1}
W20260810 07:20:24.406685 12651 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:45317 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
I20260810 07:20:24.407115 12749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.406692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.193:47847)
0810 07:20:24.406941 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.407012 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":70,"threads_started":1}
W20260810 07:20:24.407462 12254 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:45317 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
I20260810 07:20:24.919337 12750 raft_consensus.cc:482] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94a50c31fbb042299f5f43ac5b7c8755)
I20260810 07:20:24.919593 12751 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94a50c31fbb042299f5f43ac5b7c8755)
I20260810 07:20:24.919612 12750 raft_consensus.cc:504] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } }
I20260810 07:20:24.919777 12751 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } }
I20260810 07:20:24.920738 12750 leader_election.cc:288] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94a50c31fbb042299f5f43ac5b7c8755 (127.1.125.254:45317), a5b3b5cc29de475c85bdd203ee9023f5 (127.1.125.252:37325)
I20260810 07:20:24.920825 12751 leader_election.cc:288] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94a50c31fbb042299f5f43ac5b7c8755 (127.1.125.254:45317), e62a6dfd26a74cb7ba4d14bac5f9c360 (127.1.125.253:34723)
I20260810 07:20:24.921432 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.920817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45366)
0810 07:20:24.921217 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.921317 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":76,"threads_started":1}
I20260810 07:20:24.921535 12756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.920892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45374)
0810 07:20:24.921348 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.921412 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":71,"threads_started":1}
W20260810 07:20:24.921633 11670 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
W20260810 07:20:24.921674 11962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
I20260810 07:20:24.922726 12756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.922444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45398)
0810 07:20:24.922603 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.922657 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 07:20:24.922742 12769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.922231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45388)
0810 07:20:24.922617 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.922665 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":85,"threads_started":1}
W20260810 07:20:24.922860 11962 leader_election.cc:334] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 94a50c31fbb042299f5f43ac5b7c8755 (127.1.125.254:45317): Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
W20260810 07:20:24.922986 11670 leader_election.cc:334] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 94a50c31fbb042299f5f43ac5b7c8755 (127.1.125.254:45317): Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
I20260810 07:20:24.926329 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.921707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:37325 (local address 127.0.0.1:41738)
0810 07:20:24.921889 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.921904 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:24.922002 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:24.922315 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:24.922318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:24.922343 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:24.922434 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:24.922441 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.923662 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.923665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:24.924998 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:24.925011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.925879 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.925882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:24.925991 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:24.926014 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:20:24.926103 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 07:20:24.926170 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":1,"spinlock_wait_cycles":3712}
I20260810 07:20:24.926344 12761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.921355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41738 (local address 127.1.125.252:37325)
0810 07:20:24.922104 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:24.922110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:24.922122 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:24.922155 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:24.922158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:24.922199 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:24.922550 (+ 351us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:24.922645 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.923535 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.925135 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.925748 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.925864 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:24.925944 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:24.926127 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 07:20:24.926198 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":674,"thread_start_us":65,"threads_started":1}
I20260810 07:20:24.926348 12760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.921207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46186 (local address 127.1.125.253:34723)
0810 07:20:24.921688 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:24.921693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:24.921708 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:20:24.922155 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:24.922158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:24.922199 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:24.922278 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:24.922672 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.923535 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.925135 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.925748 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.925864 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:24.926005 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:24.926118 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 07:20:24.926198 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":419,"spinlock_wait_cycles":1408,"thread_start_us":74,"threads_started":1}
I20260810 07:20:24.926357 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.921064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:34723 (local address 127.0.0.1:46186)
0810 07:20:24.921435 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.921449 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:24.922006 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:24.922416 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:24.922419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:24.922434 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:24.922550 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:24.922558 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.923662 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.923665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:24.924998 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:24.925011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:24.925879 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:24.925882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:24.925978 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:24.926004 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:20:24.926090 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 07:20:24.926167 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":66,"threads_started":1}
I20260810 07:20:24.926913 11983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" is_pre_election: true
I20260810 07:20:24.927125 11983 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e62a6dfd26a74cb7ba4d14bac5f9c360 in term 1.
I20260810 07:20:24.927134 11710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" is_pre_election: true
I20260810 07:20:24.927328 11710 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5b3b5cc29de475c85bdd203ee9023f5 in term 1.
I20260810 07:20:24.927486 11669 leader_election.cc:302] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a5b3b5cc29de475c85bdd203ee9023f5, e62a6dfd26a74cb7ba4d14bac5f9c360; no voters: 94a50c31fbb042299f5f43ac5b7c8755
I20260810 07:20:24.927713 12750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:20:24.927772 12750 raft_consensus.cc:482] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 94a50c31fbb042299f5f43ac5b7c8755)
I20260810 07:20:24.927811 12750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:20:24.927760 11958 leader_election.cc:302] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a5b3b5cc29de475c85bdd203ee9023f5, e62a6dfd26a74cb7ba4d14bac5f9c360; no voters: 94a50c31fbb042299f5f43ac5b7c8755
I20260810 07:20:24.927978 12751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:20:24.928040 12751 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 94a50c31fbb042299f5f43ac5b7c8755)
I20260810 07:20:24.928095 12751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:20:24.929018 12750 raft_consensus.cc:504] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } }
I20260810 07:20:24.929203 12751 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } }
I20260810 07:20:24.929399 12750 leader_election.cc:288] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [CANDIDATE]: Term 2 election: Requested vote from peers 94a50c31fbb042299f5f43ac5b7c8755 (127.1.125.254:45317), a5b3b5cc29de475c85bdd203ee9023f5 (127.1.125.252:37325)
I20260810 07:20:24.929589 12751 leader_election.cc:288] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [CANDIDATE]: Term 2 election: Requested vote from peers 94a50c31fbb042299f5f43ac5b7c8755 (127.1.125.254:45317), e62a6dfd26a74cb7ba4d14bac5f9c360 (127.1.125.253:34723)
I20260810 07:20:24.929726 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.929512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45408)
0810 07:20:24.929603 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.929665 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 07:20:24.929728 11983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a5b3b5cc29de475c85bdd203ee9023f5"
I20260810 07:20:24.929885 11983 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e62a6dfd26a74cb7ba4d14bac5f9c360 in current term 2: Already voted for candidate a5b3b5cc29de475c85bdd203ee9023f5 in this term.
I20260810 07:20:24.929920 11710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360"
I20260810 07:20:24.930051 11710 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a5b3b5cc29de475c85bdd203ee9023f5 in current term 2: Already voted for candidate e62a6dfd26a74cb7ba4d14bac5f9c360 in this term.
I20260810 07:20:24.930691 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.930447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45432)
0810 07:20:24.930575 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.930632 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 07:20:24.930691 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.929799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45418)
0810 07:20:24.930575 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.930632 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":712}
W20260810 07:20:24.930948 11670 leader_election.cc:334] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 94a50c31fbb042299f5f43ac5b7c8755 (127.1.125.254:45317): Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
I20260810 07:20:24.931036 11670 leader_election.cc:302] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e62a6dfd26a74cb7ba4d14bac5f9c360; no voters: 94a50c31fbb042299f5f43ac5b7c8755, a5b3b5cc29de475c85bdd203ee9023f5
I20260810 07:20:24.931200 12750 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 07:20:24.931420 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:24.931158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45436)
0810 07:20:24.931301 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:24.931358 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 07:20:24.931527 11962 leader_election.cc:334] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 94a50c31fbb042299f5f43ac5b7c8755 (127.1.125.254:45317): Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
I20260810 07:20:24.931607 11962 leader_election.cc:302] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5b3b5cc29de475c85bdd203ee9023f5; no voters: 94a50c31fbb042299f5f43ac5b7c8755, e62a6dfd26a74cb7ba4d14bac5f9c360
I20260810 07:20:24.931732 12751 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 07:20:25.407064 12770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.406448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.194:37851)
0810 07:20:25.406870 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:25.406948 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":128,"threads_started":1}
I20260810 07:20:25.408095 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.407457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.195:52047)
0810 07:20:25.407851 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:25.407932 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":84,"threads_started":1}
I20260810 07:20:25.408567 12772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:25.408004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.193:51341)
0810 07:20:25.408265 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:25.408406 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":75,"threads_started":1}
I20260810 07:20:26.408721 12773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.407926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.194:51269)
0810 07:20:26.408507 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.408586 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":86,"threads_started":1}
I20260810 07:20:26.409770 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.409201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.193:34573)
0810 07:20:26.409602 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.409665 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":66,"threads_started":1}
I20260810 07:20:26.409776 12774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.408882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.195:39637)
0810 07:20:26.409616 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.409680 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":81,"threads_started":1}
I20260810 07:20:26.525398 12777 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:26.525542 12777 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } }
I20260810 07:20:26.526211 12777 leader_election.cc:288] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94a50c31fbb042299f5f43ac5b7c8755 (127.1.125.254:45317), e62a6dfd26a74cb7ba4d14bac5f9c360 (127.1.125.253:34723)
I20260810 07:20:26.526723 11710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" is_pre_election: true
I20260810 07:20:26.526921 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.526447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45668)
0810 07:20:26.526718 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.526802 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"threads_started":1}
I20260810 07:20:26.526922 11710 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5b3b5cc29de475c85bdd203ee9023f5 in term 2.
I20260810 07:20:26.527303 11958 leader_election.cc:302] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5b3b5cc29de475c85bdd203ee9023f5, e62a6dfd26a74cb7ba4d14bac5f9c360; no voters:
I20260810 07:20:26.527539 12777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 07:20:26.527628 12777 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:26.527696 12777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:20:26.528172 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.527866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45684)
0810 07:20:26.528020 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.528099 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 07:20:26.528311 11962 leader_election.cc:334] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 94a50c31fbb042299f5f43ac5b7c8755 (127.1.125.254:45317): Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
I20260810 07:20:26.529203 12777 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } }
I20260810 07:20:26.529628 12777 leader_election.cc:288] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [CANDIDATE]: Term 3 election: Requested vote from peers 94a50c31fbb042299f5f43ac5b7c8755 (127.1.125.254:45317), e62a6dfd26a74cb7ba4d14bac5f9c360 (127.1.125.253:34723)
I20260810 07:20:26.530006 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.529716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45696)
0810 07:20:26.529897 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.529948 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260810 07:20:26.530061 11710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360"
I20260810 07:20:26.530211 11710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:20:26.530791 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.530546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45702)
0810 07:20:26.530669 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.530735 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 07:20:26.530926 11962 leader_election.cc:334] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 94a50c31fbb042299f5f43ac5b7c8755 (127.1.125.254:45317): Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
I20260810 07:20:26.531378 11710 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5b3b5cc29de475c85bdd203ee9023f5 in term 3.
I20260810 07:20:26.531769 11958 leader_election.cc:302] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a5b3b5cc29de475c85bdd203ee9023f5, e62a6dfd26a74cb7ba4d14bac5f9c360; no voters: 94a50c31fbb042299f5f43ac5b7c8755
I20260810 07:20:26.531955 12777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 07:20:26.532222 12777 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [term 3 LEADER]: Becoming Leader. State: Replica: a5b3b5cc29de475c85bdd203ee9023f5, State: Running, Role: LEADER
I20260810 07:20:26.532547 12777 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } }
I20260810 07:20:26.534060 12781 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5b3b5cc29de475c85bdd203ee9023f5. Latest consensus state: current_term: 3 leader_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } } }
I20260810 07:20:26.534206 12781 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:26.534538 12784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:26.536720 12784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:26.537068 12784 catalog_manager.cc:1259] Loaded cluster ID: da4a4ebf438b41e1bb371235fd054f0f
I20260810 07:20:26.537109 12784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:26.537626 12784 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:26.537976 12784 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: Loaded TSK: 0
I20260810 07:20:26.538616 12784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:20:26.600878 11978 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:20:26.604065 11978 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:26.607369 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.607070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45714)
0810 07:20:26.607206 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.607284 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 07:20:26.607579 11710 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [term 3 FOLLOWER]: Refusing update from remote peer a5b3b5cc29de475c85bdd203ee9023f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260810 07:20:26.608165 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.607890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.0.0.1:45718)
0810 07:20:26.608013 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.608074 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 07:20:26.608148 12781 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 07:20:26.608335 11962 consensus_peers.cc:487] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 -> Peer 94a50c31fbb042299f5f43ac5b7c8755 (127.1.125.254:45317): Couldn't send request to peer 94a50c31fbb042299f5f43ac5b7c8755. Status: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 07:20:26.609819 12786 sys_catalog.cc:455] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5b3b5cc29de475c85bdd203ee9023f5. Latest consensus state: current_term: 3 leader_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } } }
I20260810 07:20:26.609939 12786 sys_catalog.cc:458] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:20:26.611020 12781 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } } }
I20260810 07:20:26.611164 12781 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:26.611601 12786 sys_catalog.cc:455] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "94a50c31fbb042299f5f43ac5b7c8755" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45317 } } peers { permanent_uuid: "e62a6dfd26a74cb7ba4d14bac5f9c360" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 34723 } attrs { promote: false } } peers { permanent_uuid: "a5b3b5cc29de475c85bdd203ee9023f5" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 37325 } attrs { promote: false } } }
I20260810 07:20:26.611719 12786 sys_catalog.cc:458] T 00000000000000000000000000000000 P e62a6dfd26a74cb7ba4d14bac5f9c360 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:20:26.622207 12795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.617913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:44005 (local address 127.0.0.1:55014)
0810 07:20:26.618413 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.618429 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:26.618521 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:26.618952 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:26.618955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:26.618980 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:26.619072 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:26.619077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.620045 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.620050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:26.620960 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:26.620971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.621695 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.621699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:26.621810 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:26.621859 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 07:20:26.621949 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 07:20:26.622038 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":89,"threads_started":1}
I20260810 07:20:26.622840 12796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.618030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35852 (local address 127.1.125.193:33107)
0810 07:20:26.618292 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:26.618297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:26.618311 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:20:26.618370 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:26.618376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:26.618434 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:26.618873 (+ 439us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:26.619006 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.619769 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.621271 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.621900 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.622495 (+ 595us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:26.622558 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:26.622615 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 07:20:26.622707 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":85,"threads_started":1}
I20260810 07:20:26.622932 12797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.618285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55014 (local address 127.1.125.195:44005)
0810 07:20:26.618549 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:26.618554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:26.618637 (+ 83us) server_negotiation.cc:407] Connection header received
0810 07:20:26.618695 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:26.618698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:26.618747 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:26.618846 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:26.619234 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.619911 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.621112 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.621551 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.621697 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:26.621774 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:26.622700 (+ 926us) server_negotiation.cc:299] Negotiation successful
0810 07:20:26.622781 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
I20260810 07:20:26.622387 12793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.617569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:33107 (local address 127.0.0.1:35852)
0810 07:20:26.618120 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.618135 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:26.618228 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:26.618573 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:26.618588 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:26.618609 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:26.618755 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:26.618762 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.619903 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.619907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:26.621130 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:26.621141 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.622031 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.622035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:26.622120 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:26.622146 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:20:26.622193 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 07:20:26.622255 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":103,"threads_started":1}
I20260810 07:20:26.623155 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.617293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:33527 (local address 127.0.0.1:40070)
0810 07:20:26.617428 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.617442 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:26.617553 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:26.619326 (+ 1773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:26.619329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:26.619352 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:26.619451 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:26.619459 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.620520 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.620524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:26.621617 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:26.621627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.622746 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.622750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:26.622837 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:26.622871 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 07:20:26.622930 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:20:26.623005 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 07:20:26.623780 12794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.617769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40070 (local address 127.1.125.194:33527)
0810 07:20:26.619092 (+ 1323us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:26.619097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:26.619110 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:20:26.619162 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:26.619165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:26.619213 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:26.619298 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:26.619591 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.620358 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.621914 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.622611 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.623477 (+ 866us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:26.623529 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:26.623560 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 07:20:26.623652 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1225,"thread_start_us":99,"threads_started":1}
I20260810 07:20:26.623916 12547 tablet_service.cc:1467] Processing CreateTablet for tablet f52f97e1b98344469f0f834ad161f235 (DEFAULT_TABLE table=test-table [id=9feea8d2240f4ec2a48e44df1de0f5e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:26.624104 12152 tablet_service.cc:1467] Processing CreateTablet for tablet f52f97e1b98344469f0f834ad161f235 (DEFAULT_TABLE table=test-table [id=9feea8d2240f4ec2a48e44df1de0f5e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:26.624251 12547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f52f97e1b98344469f0f834ad161f235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:26.624336 12354 tablet_service.cc:1467] Processing CreateTablet for tablet f52f97e1b98344469f0f834ad161f235 (DEFAULT_TABLE table=test-table [id=9feea8d2240f4ec2a48e44df1de0f5e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:26.624421 12152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f52f97e1b98344469f0f834ad161f235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:26.624652 12354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f52f97e1b98344469f0f834ad161f235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:26.626767 12799 tablet_bootstrap.cc:492] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8: Bootstrap starting.
I20260810 07:20:26.627264 12800 tablet_bootstrap.cc:492] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923: Bootstrap starting.
I20260810 07:20:26.627815 12801 tablet_bootstrap.cc:492] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e: Bootstrap starting.
I20260810 07:20:26.628113 12799 tablet_bootstrap.cc:654] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:26.628258 12800 tablet_bootstrap.cc:654] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:26.628813 12801 tablet_bootstrap.cc:654] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:26.629904 12799 tablet_bootstrap.cc:492] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8: No bootstrap required, opened a new log
I20260810 07:20:26.630007 12799 ts_tablet_manager.cc:1397] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:20:26.630016 12800 tablet_bootstrap.cc:492] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923: No bootstrap required, opened a new log
I20260810 07:20:26.630112 12800 ts_tablet_manager.cc:1397] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:20:26.630342 12801 tablet_bootstrap.cc:492] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e: No bootstrap required, opened a new log
I20260810 07:20:26.630439 12801 ts_tablet_manager.cc:1397] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:20:26.630482 12799 raft_consensus.cc:348] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:26.630631 12799 raft_consensus.cc:374] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:26.630671 12799 raft_consensus.cc:729] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1bb10376c747af8a925456ebb1c4a8, State: Initialized, Role: FOLLOWER
I20260810 07:20:26.630638 12800 raft_consensus.cc:348] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:26.630764 12800 raft_consensus.cc:374] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:26.630813 12800 raft_consensus.cc:729] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06f27bfb4699425ead7908ac41157923, State: Initialized, Role: FOLLOWER
I20260810 07:20:26.630848 12799 consensus_queue.cc:260] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:26.630939 12801 raft_consensus.cc:348] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:26.630959 12800 consensus_queue.cc:260] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:26.631057 12801 raft_consensus.cc:374] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:26.631093 12801 raft_consensus.cc:729] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce643b0a17074c9e9a974c4ac36f711e, State: Initialized, Role: FOLLOWER
I20260810 07:20:26.631273 12799 ts_tablet_manager.cc:1428] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260810 07:20:26.631268 12801 consensus_queue.cc:260] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:26.631412 12800 ts_tablet_manager.cc:1428] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260810 07:20:26.632187 12774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.631692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.195:38927)
0810 07:20:26.632039 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.632109 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254}
I20260810 07:20:26.632381 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.632102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.193:40527)
0810 07:20:26.632264 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.632318 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 07:20:26.633562 12774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.633308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.195:49959)
0810 07:20:26.633471 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.633513 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 07:20:26.634068 12648 heartbeater.cc:503] Master 127.1.125.252:37325 was elected leader, sending a full tablet report...
I20260810 07:20:26.634212 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.633924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.193:39669)
0810 07:20:26.634093 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.634148 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 07:20:26.636435 12253 heartbeater.cc:503] Master 127.1.125.252:37325 was elected leader, sending a full tablet report...
I20260810 07:20:26.636612 12801 ts_tablet_manager.cc:1428] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260810 07:20:26.637761 12773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.637248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.194:32805)
0810 07:20:26.637617 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.637668 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284}
I20260810 07:20:26.638200 12446 heartbeater.cc:503] Master 127.1.125.252:37325 was elected leader, sending a full tablet report...
I20260810 07:20:26.638510 12811 raft_consensus.cc:482] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:26.638664 12773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.638308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.194:54621)
0810 07:20:26.638529 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.638593 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260810 07:20:26.638636 12811 raft_consensus.cc:504] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:26.639904 12811 leader_election.cc:288] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06f27bfb4699425ead7908ac41157923 (127.1.125.193:33107), 3f1bb10376c747af8a925456ebb1c4a8 (127.1.125.195:44005)
I20260810 07:20:26.644515 12797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.640182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:49035 (local address 127.1.125.195:44005)
0810 07:20:26.640298 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:26.640303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:26.640766 (+ 463us) server_negotiation.cc:407] Connection header received
0810 07:20:26.640939 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:26.640942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:26.640986 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:26.641038 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:26.641481 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.641973 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.643310 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.643968 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.644085 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:26.644128 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:26.644319 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 07:20:26.644394 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:20:26.644536 12812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.640069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:44005 (local address 127.1.125.194:49035)
0810 07:20:26.640678 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.640692 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:26.640805 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:26.641195 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:26.641198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:26.641217 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:26.641377 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:26.641387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.642098 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.642101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:26.643184 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:26.643196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.644106 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.644110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:26.644182 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:26.644203 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:20:26.644299 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 07:20:26.644369 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"mutex_wait_us":68,"thread_start_us":137,"threads_started":1}
I20260810 07:20:26.645000 12577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f52f97e1b98344469f0f834ad161f235" candidate_uuid: "ce643b0a17074c9e9a974c4ac36f711e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1bb10376c747af8a925456ebb1c4a8" is_pre_election: true
I20260810 07:20:26.645190 12577 raft_consensus.cc:2454] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce643b0a17074c9e9a974c4ac36f711e in term 0.
I20260810 07:20:26.645277 12773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.640015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:33107 (local address 127.1.125.194:42237)
0810 07:20:26.640403 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.640425 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:26.640569 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:26.640850 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:26.640853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:26.640872 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:26.641021 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:26.641030 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.641934 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.641940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:26.643184 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:26.643196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.644862 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.644866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:26.644976 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:26.644998 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:20:26.645073 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:20:26.645123 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":74}
I20260810 07:20:26.645316 12796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.640292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:42237 (local address 127.1.125.193:33107)
0810 07:20:26.640393 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:26.640397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:26.640516 (+ 119us) server_negotiation.cc:407] Connection header received
0810 07:20:26.640697 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:26.640700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:26.640751 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:26.640834 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:26.641203 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.641798 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.644184 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.644733 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.644878 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:26.644935 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:26.645116 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 07:20:26.645200 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 07:20:26.645650 12284 leader_election.cc:302] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1bb10376c747af8a925456ebb1c4a8, ce643b0a17074c9e9a974c4ac36f711e; no voters:
I20260810 07:20:26.645934 12811 raft_consensus.cc:2781] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:26.645998 12811 raft_consensus.cc:482] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:26.646030 12811 raft_consensus.cc:3037] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:26.647054 12811 raft_consensus.cc:504] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:26.647397 12811 leader_election.cc:288] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [CANDIDATE]: Term 1 election: Requested vote from peers 06f27bfb4699425ead7908ac41157923 (127.1.125.193:33107), 3f1bb10376c747af8a925456ebb1c4a8 (127.1.125.195:44005)
I20260810 07:20:26.647938 12179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f52f97e1b98344469f0f834ad161f235" candidate_uuid: "ce643b0a17074c9e9a974c4ac36f711e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06f27bfb4699425ead7908ac41157923"
I20260810 07:20:26.648171 12179 raft_consensus.cc:3037] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:26.648689 12577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f52f97e1b98344469f0f834ad161f235" candidate_uuid: "ce643b0a17074c9e9a974c4ac36f711e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1bb10376c747af8a925456ebb1c4a8"
I20260810 07:20:26.648803 12577 raft_consensus.cc:3037] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:26.650048 12577 raft_consensus.cc:2454] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce643b0a17074c9e9a974c4ac36f711e in term 1.
I20260810 07:20:26.650519 12284 leader_election.cc:302] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1bb10376c747af8a925456ebb1c4a8, ce643b0a17074c9e9a974c4ac36f711e; no voters:
I20260810 07:20:26.650619 12180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f52f97e1b98344469f0f834ad161f235" candidate_uuid: "ce643b0a17074c9e9a974c4ac36f711e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06f27bfb4699425ead7908ac41157923" is_pre_election: true
I20260810 07:20:26.651707 12179 raft_consensus.cc:2454] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce643b0a17074c9e9a974c4ac36f711e in term 1.
I20260810 07:20:26.652770 12811 raft_consensus.cc:2781] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:26.653010 12811 raft_consensus.cc:686] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [term 1 LEADER]: Becoming Leader. State: Replica: ce643b0a17074c9e9a974c4ac36f711e, State: Running, Role: LEADER
I20260810 07:20:26.653270 12811 consensus_queue.cc:237] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } }
I20260810 07:20:26.655354 12773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.655017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.194:46507)
0810 07:20:26.655168 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.655244 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 07:20:26.657174 11978 catalog_manager.cc:5495] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e reported cstate change: term changed from 0 to 1, leader changed from <none> to ce643b0a17074c9e9a974c4ac36f711e (127.1.125.194). New cstate: current_term: 1 leader_uuid: "ce643b0a17074c9e9a974c4ac36f711e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce643b0a17074c9e9a974c4ac36f711e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 33527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 } health_report { overall_health: UNKNOWN } } }
W20260810 07:20:26.659319 12256 tablet.cc:2369] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:20:26.674810 12794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.671459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40080 (local address 127.1.125.194:33527)
0810 07:20:26.671623 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:26.671627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:26.671640 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:20:26.671778 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:26.671780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:26.671832 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:26.671920 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:26.672150 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.672724 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.673546 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:26.673756 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:26.673836 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:26.674074 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:26.674507 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:26.674595 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:26.674657 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 07:20:26.674705 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 07:20:26.677616 12577 raft_consensus.cc:1261] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8 [term 1 FOLLOWER]: Refusing update from remote peer ce643b0a17074c9e9a974c4ac36f711e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:26.677879 12179 raft_consensus.cc:1261] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923 [term 1 FOLLOWER]: Refusing update from remote peer ce643b0a17074c9e9a974c4ac36f711e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:26.678145 12811 consensus_queue.cc:1035] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1bb10376c747af8a925456ebb1c4a8" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 44005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:26.678391 12813 consensus_queue.cc:1035] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [LEADER]: Connected to new peer: Peer: permanent_uuid: "06f27bfb4699425ead7908ac41157923" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:26.680342 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.680018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.193:39291)
0810 07:20:26.680203 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.680262 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260810 07:20:26.681151 12774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.680858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.195:54549)
0810 07:20:26.681016 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.681075 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 07:20:26.682938 12773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.682672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.194:37789)
0810 07:20:26.682818 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.682869 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 07:20:26.683620 12823 mvcc.cc:205] Tried to move back new op lower bound from 7316874963666329600 to 7316874963574476800. Current Snapshot: MvccSnapshot[applied={T|T < 7316874963666329600}]
I20260810 07:20:26.684506 12774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.684213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.195:51187)
0810 07:20:26.684344 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.684403 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 07:20:26.686103 12773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.685880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.194:38319)
0810 07:20:26.685975 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.686026 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 07:20:26.686990 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.686760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45317 (local address 127.1.125.193:35235)
0810 07:20:26.686866 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.686923 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 07:20:26.687049 12825 mvcc.cc:205] Tried to move back new op lower bound from 7316874963666329600 to 7316874963574476800. Current Snapshot: MvccSnapshot[applied={T|T < 7316874963666329600}]
I20260810 07:20:26.694363 11977 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:26.694502 11977 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:26.698340 11977 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5b3b5cc29de475c85bdd203ee9023f5: Sending DeleteTablet for 3 replicas of tablet f52f97e1b98344469f0f834ad161f235
I20260810 07:20:26.699198 12152 tablet_service.cc:1514] Processing DeleteTablet for tablet f52f97e1b98344469f0f834ad161f235 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:26 UTC) from {username='oryx', principal='oryx/127.1.125.252@KRBTEST.COM'} at 127.0.0.1:35852
I20260810 07:20:26.699257 12354 tablet_service.cc:1514] Processing DeleteTablet for tablet f52f97e1b98344469f0f834ad161f235 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:26 UTC) from {username='oryx', principal='oryx/127.1.125.252@KRBTEST.COM'} at 127.0.0.1:40070
I20260810 07:20:26.699554 12835 tablet_replica.cc:330] T f52f97e1b98344469f0f834ad161f235 P 06f27bfb4699425ead7908ac41157923: stopping tablet replica
I20260810 07:20:26.699527 12547 tablet_service.cc:1514] Processing DeleteTablet for tablet f52f97e1b98344469f0f834ad161f235 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:26 UTC) from {username='oryx', principal='oryx/127.1.125.252@KRBTEST.COM'} at 127.0.0.1:55014
I20260810 07:20:26.699608 12836 tablet_replica.cc:330] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e: stopping tablet replica
I20260810 07:20:26.699731 12836 raft_consensus.cc:2229] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:26.699888 12837 tablet_replica.cc:330] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8: stopping tablet replica
I20260810 07:20:26.700027 12837 raft_consensus.cc:2229] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:26.700034 12836 raft_consensus.cc:2258] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:26.700206 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 11912
I20260810 07:20:26.700304 12837 raft_consensus.cc:2258] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:26.707576 12836 ts_tablet_manager.cc:1905] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:26.710665 12836 ts_tablet_manager.cc:1918] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:26.710757 12836 log.cc:1196] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/f52f97e1b98344469f0f834ad161f235
I20260810 07:20:26.711056 12836 ts_tablet_manager.cc:1939] T f52f97e1b98344469f0f834ad161f235 P ce643b0a17074c9e9a974c4ac36f711e: Deleting consensus metadata
I20260810 07:20:26.711880 11957 catalog_manager.cc:4841] TS ce643b0a17074c9e9a974c4ac36f711e (127.1.125.194:33527): tablet f52f97e1b98344469f0f834ad161f235 (table test-table [id=9feea8d2240f4ec2a48e44df1de0f5e4]) successfully deleted
I20260810 07:20:26.717181 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 12261
I20260810 07:20:26.717537 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:26.717230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:33107 (local address 127.0.0.1:35864)
0810 07:20:26.717406 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:26.717455 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:33107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260810 07:20:26.724166 11962 catalog_manager.cc:4584] TS 06f27bfb4699425ead7908ac41157923 (127.1.125.193:33107): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f52f97e1b98344469f0f834ad161f235: Network error: Client connection negotiation failed: client connection to 127.1.125.193:33107: connect: Connection refused (error 111)
I20260810 07:20:26.731446 12837 ts_tablet_manager.cc:1905] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:26.734845 12837 ts_tablet_manager.cc:1918] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:26.734926 12837 log.cc:1196] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/f52f97e1b98344469f0f834ad161f235
I20260810 07:20:26.735219 12837 ts_tablet_manager.cc:1939] T f52f97e1b98344469f0f834ad161f235 P 3f1bb10376c747af8a925456ebb1c4a8: Deleting consensus metadata
I20260810 07:20:26.736090 11953 catalog_manager.cc:4841] TS 3f1bb10376c747af8a925456ebb1c4a8 (127.1.125.195:44005): tablet f52f97e1b98344469f0f834ad161f235 (table test-table [id=9feea8d2240f4ec2a48e44df1de0f5e4]) successfully deleted
I20260810 07:20:26.752933 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 12457
I20260810 07:20:26.766165 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 11491
I20260810 07:20:26.777714 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 11644
2026-08-10T07:20:26Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7494 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:26 dist-test-slave-44d2 krb5kdc[12841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:26 dist-test-slave-44d2 krb5kdc[12841](info): set up 2 sockets
Aug 10 07:20:26 dist-test-slave-44d2 krb5kdc[12841](info): commencing operation
krb5kdc: starting...
W20260810 07:20:28.085258 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:20:28 dist-test-slave-44d2 krb5kdc[12841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:28Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.254.keytab.
Entry for principal oryx/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.254.keytab.
I20260810 07:20:28.217927 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:34167
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.254.keytab
--principal=oryx/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:45797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:34167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:28.334446 12860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:28.334705 12860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:28.334748 12860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:28.338688 12860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:28.338738 12860 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 07:20:28.338757 12860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:28.338773 12860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:28.338789 12860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:28.338804 12860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:28.343957 12860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:34167
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.254.keytab
--principal=oryx/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:34167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:28.345041 12860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:28.345973 12860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:20:28.355087 12860 server_base.cc:1034] running on GCE node
W20260810 07:20:28.355106 12871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:28.355278 12869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:28.355262 12874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:28.356209 12860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:28.357376 12860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:28.358543 12860 hybrid_clock.cc:648] HybridClock initialized: now 1786346428358517 us; error 47 us; skew 500 ppm
Aug 10 07:20:28 dist-test-slave-44d2 krb5kdc[12841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346428, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:28.361421 12860 init.cc:377] Logged in from keytab as oryx/127.1.125.254@KRBTEST.COM (short username oryx)
I20260810 07:20:28.362502 12860 webserver.cc:460] Webserver started at http://127.1.125.254:36053/ using document root <none> and password file <none>
I20260810 07:20:28.363153 12860 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:28.363260 12860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:28.363520 12860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:28.366202 12860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "18e430a9a9734a9ba37b4cdd4ca8b83d"
format_stamp: "Formatted at 2026-08-10 07:20:28 on dist-test-slave-44d2"
server_key: "bc1e22d7ec671f3278145bf9be483555"
server_key_iv: "72e338dd8e8f84c266666fc740f854a0"
server_key_version: "encryptionkey@0"
I20260810 07:20:28.366904 12860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "18e430a9a9734a9ba37b4cdd4ca8b83d"
format_stamp: "Formatted at 2026-08-10 07:20:28 on dist-test-slave-44d2"
server_key: "bc1e22d7ec671f3278145bf9be483555"
server_key_iv: "72e338dd8e8f84c266666fc740f854a0"
server_key_version: "encryptionkey@0"
I20260810 07:20:28.371296 12860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 07:20:28.374245 12883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:28.375705 12860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:20:28.375877 12860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "18e430a9a9734a9ba37b4cdd4ca8b83d"
format_stamp: "Formatted at 2026-08-10 07:20:28 on dist-test-slave-44d2"
server_key: "bc1e22d7ec671f3278145bf9be483555"
server_key_iv: "72e338dd8e8f84c266666fc740f854a0"
server_key_version: "encryptionkey@0"
I20260810 07:20:28.376003 12860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:28.394810 12860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:28.397959 12860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:28.398180 12860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:28.406555 12860 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:34167
I20260810 07:20:28.406572 12969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:34167 every 8 connection(s)
I20260810 07:20:28.407588 12860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:28.410454 12970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:28.415207 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 12860
I20260810 07:20:28.415361 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:28.415720 1527 external_mini_cluster.cc:1428] Setting key 963408fdc64d3518523e71d394621f7f
I20260810 07:20:28.416167 12970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d: Bootstrap starting.
I20260810 07:20:28.418883 12970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:28.419806 12970 log.cc:824] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:28.422106 12970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d: No bootstrap required, opened a new log
Aug 10 07:20:28 dist-test-slave-44d2 krb5kdc[12841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:28.425709 12970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e430a9a9734a9ba37b4cdd4ca8b83d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34167 } }
I20260810 07:20:28.426012 12970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:28.426101 12970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e430a9a9734a9ba37b4cdd4ca8b83d, State: Initialized, Role: FOLLOWER
I20260810 07:20:28.426613 12970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e430a9a9734a9ba37b4cdd4ca8b83d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34167 } }
I20260810 07:20:28.426807 12970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:28.426884 12970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:28.427009 12970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:28.427871 12970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e430a9a9734a9ba37b4cdd4ca8b83d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34167 } }
I20260810 07:20:28.428220 12970 leader_election.cc:302] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18e430a9a9734a9ba37b4cdd4ca8b83d; no voters:
I20260810 07:20:28.428581 12970 leader_election.cc:288] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:28.428655 12980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:28.428874 12980 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [term 1 LEADER]: Becoming Leader. State: Replica: 18e430a9a9734a9ba37b4cdd4ca8b83d, State: Running, Role: LEADER
I20260810 07:20:28.429172 12980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e430a9a9734a9ba37b4cdd4ca8b83d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34167 } }
I20260810 07:20:28.429930 12970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:28.430868 12981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18e430a9a9734a9ba37b4cdd4ca8b83d. Latest consensus state: current_term: 1 leader_uuid: "18e430a9a9734a9ba37b4cdd4ca8b83d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e430a9a9734a9ba37b4cdd4ca8b83d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34167 } } }
I20260810 07:20:28.431023 12981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:28.431371 12982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18e430a9a9734a9ba37b4cdd4ca8b83d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e430a9a9734a9ba37b4cdd4ca8b83d" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34167 } } }
I20260810 07:20:28.431442 12993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:28.431465 12982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:28.432366 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:28.417230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37714 (local address 127.1.125.254:34167)
0810 07:20:28.417668 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:28.417688 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:28.417731 (+ 43us) server_negotiation.cc:407] Connection header received
0810 07:20:28.418540 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:28.418564 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:28.419010 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:28.419394 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:28.420151 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.421589 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.422537 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.423208 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.423821 (+ 613us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:28.425949 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:28.425955 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:28.425973 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:28.426007 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:28.427935 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:28.428516 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:28.428522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:28.428528 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:28.428693 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:28.429106 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:28.429109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:28.429111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:28.429597 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:28.429734 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:28.430688 (+ 954us) server_negotiation.cc:299] Negotiation successful
0810 07:20:28.431031 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":108,"threads_started":1}
I20260810 07:20:28.436757 12993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:28.442220 12993 catalog_manager.cc:1347] Generated new cluster ID: 0dcb10083fb948feb014b8b0f457ab4e
I20260810 07:20:28.442287 12993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:28.449575 12993 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:28.450745 12993 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:28.456101 12993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d: Generated new TSK 0
I20260810 07:20:28.456806 12993 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab.
Entry for principal oryx/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab.
I20260810 07:20:28.495330 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:34167
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab
--principal=oryx/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:45797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:28.622668 13016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:28.622953 13016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:28.623039 13016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:28.627431 13016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:28.627488 13016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:28.627563 13016 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:20:28.627646 13016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:28.633247 13016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab
--principal=oryx/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.125.254:34167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:28.634275 13016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:28.635313 13016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:28.645207 13021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:28.645267 13023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:28.645305 13016 server_base.cc:1034] running on GCE node
W20260810 07:20:28.645272 13027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:28.649031 13016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:28.649917 13016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:28.651113 13016 hybrid_clock.cc:648] HybridClock initialized: now 1786346428651086 us; error 46 us; skew 500 ppm
Aug 10 07:20:28 dist-test-slave-44d2 krb5kdc[12841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346428, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:28.657361 13016 init.cc:377] Logged in from keytab as oryx/127.1.125.193@KRBTEST.COM (short username oryx)
I20260810 07:20:28.658308 13016 webserver.cc:460] Webserver started at http://127.1.125.193:42307/ using document root <none> and password file <none>
I20260810 07:20:28.658906 13016 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:28.659006 13016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:28.659246 13016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:28.661404 13016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99"
format_stamp: "Formatted at 2026-08-10 07:20:28 on dist-test-slave-44d2"
server_key: "a246c686afa2eb97079d0d81b2dff51e"
server_key_iv: "f26de627edfcecb226c31bbcd04ccad6"
server_key_version: "encryptionkey@0"
I20260810 07:20:28.661983 13016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99"
format_stamp: "Formatted at 2026-08-10 07:20:28 on dist-test-slave-44d2"
server_key: "a246c686afa2eb97079d0d81b2dff51e"
server_key_iv: "f26de627edfcecb226c31bbcd04ccad6"
server_key_version: "encryptionkey@0"
I20260810 07:20:28.666141 13016 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:28.668622 13035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:28.670045 13016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:28.670212 13016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99"
format_stamp: "Formatted at 2026-08-10 07:20:28 on dist-test-slave-44d2"
server_key: "a246c686afa2eb97079d0d81b2dff51e"
server_key_iv: "f26de627edfcecb226c31bbcd04ccad6"
server_key_version: "encryptionkey@0"
I20260810 07:20:28.670351 13016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:28.682787 13016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:28.686259 13016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:28.686504 13016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:28.688220 13016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:28.688304 13016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:28.688380 13016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:28.688444 13016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:20:28 dist-test-slave-44d2 krb5kdc[12841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346428, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:28.701341 13016 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:37879
I20260810 07:20:28.702328 13016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:28.703280 13201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:37879 every 8 connection(s)
I20260810 07:20:28.703329 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 13016
I20260810 07:20:28.703444 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:28.703684 1527 external_mini_cluster.cc:1428] Setting key 886cecac8588c1bd2db727ab98f5df34
I20260810 07:20:28.704955 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:28.688855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:57505 (local address 127.1.125.254:34167)
0810 07:20:28.689033 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:28.689039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:28.690051 (+ 1012us) server_negotiation.cc:407] Connection header received
0810 07:20:28.691428 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:28.691430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:28.691480 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:28.691555 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:28.693454 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.694198 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.695452 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.695712 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.695830 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:28.699942 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:28.699946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:28.699949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:28.699982 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:28.701713 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:28.702234 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:28.702239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:28.702240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:28.702325 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:28.702686 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:28.702691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:28.702693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:28.702958 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:28.703060 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:28.704638 (+ 1578us) server_negotiation.cc:299] Negotiation successful
0810 07:20:28.704784 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 07:20:28.705783 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:28.689309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34167 (local address 127.1.125.193:57505)
0810 07:20:28.689839 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:28.689915 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:28.691168 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:28.691762 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:28.691776 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:28.692384 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:28.693254 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:28.693281 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.694337 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.694348 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:28.695320 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:28.695332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.695874 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.695878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:28.696047 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:28.696697 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:28.696739 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:28.699762 (+ 3023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:28.701836 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:28.701843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:28.701857 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:28.702123 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:28.702422 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:28.702426 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:28.702428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:28.702575 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:28.704032 (+ 1457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:28.704038 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:28.704329 (+ 291us) client_negotiation.cc:769] Sending connection context
0810 07:20:28.704584 (+ 255us) client_negotiation.cc:240] Negotiation successful
0810 07:20:28.704856 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":105,"threads_started":1}
I20260810 07:20:28.707741 13202 heartbeater.cc:347] Connected to a master server at 127.1.125.254:34167
I20260810 07:20:28.707907 13202 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:28.708588 13202 heartbeater.cc:511] Master 127.1.125.254:34167 requested a full tablet report, sending...
I20260810 07:20:28.710294 12904 ts_manager.cc:194] Registered new tserver with Master: e6901a7f9e3e4f4d902b29bc7a63dc99 (127.1.125.193:37879)
I20260810 07:20:28.711757 12904 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:57505
WARNING: no policy specified for oryx/127.1.125.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.125.194@KRBTEST.COM" created.
I20260810 07:20:28.719126 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:28.710284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34167 (local address 127.1.125.193:42053)
0810 07:20:28.710695 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:28.710711 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:28.710818 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:28.711097 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:28.711103 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:28.711341 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:28.711532 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:28.711546 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.712731 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.712740 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:28.713680 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:28.713703 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.714474 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.714479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:28.714595 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:28.715204 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:28.715218 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:28.715559 (+ 341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:28.717405 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:28.717411 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:28.717417 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:28.717689 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:28.718030 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:28.718034 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:28.718036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:28.718134 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:28.718609 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:28.718616 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:28.718739 (+ 123us) client_negotiation.cc:769] Sending connection context
0810 07:20:28.718868 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 07:20:28.718981 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":130,"threads_started":1}
I20260810 07:20:28.719368 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:28.710368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:42053 (local address 127.1.125.254:34167)
0810 07:20:28.710496 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:28.710500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:28.710800 (+ 300us) server_negotiation.cc:407] Connection header received
0810 07:20:28.710917 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:28.710921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:28.710968 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:28.711060 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:28.711692 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.712550 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.713851 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.714207 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.714294 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:28.715723 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:28.715727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:28.715730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:28.715759 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:28.717278 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:28.717802 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:28.717807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:28.717808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:28.717912 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:28.718240 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:28.718244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:28.718246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:28.718463 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:28.718593 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:28.718978 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 07:20:28.719114 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab.
Entry for principal oryx/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab.
I20260810 07:20:28.735720 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:34167
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab
--principal=oryx/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:45797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:28.850932 13213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:28.851220 13213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:28.851296 13213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:28.855239 13213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:28.855321 13213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:28.855393 13213 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:20:28.855471 13213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:28.860620 13213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab
--principal=oryx/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.125.254:34167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:28.861637 13213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:28.862664 13213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:28.872550 13221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:28.872613 13223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:28.872612 13219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:28.872733 13213 server_base.cc:1034] running on GCE node
I20260810 07:20:28.873230 13213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:28.874010 13213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:28.875202 13213 hybrid_clock.cc:648] HybridClock initialized: now 1786346428875166 us; error 47 us; skew 500 ppm
Aug 10 07:20:28 dist-test-slave-44d2 krb5kdc[12841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346428, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:28.878098 13213 init.cc:377] Logged in from keytab as oryx/127.1.125.194@KRBTEST.COM (short username oryx)
I20260810 07:20:28.879056 13213 webserver.cc:460] Webserver started at http://127.1.125.194:44217/ using document root <none> and password file <none>
I20260810 07:20:28.879647 13213 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:28.879745 13213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:28.880008 13213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:28.882153 13213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "31e1c407fdca451f95315081115ce9c1"
format_stamp: "Formatted at 2026-08-10 07:20:28 on dist-test-slave-44d2"
server_key: "3bb345041e700c86c77b944d07afea74"
server_key_iv: "71eac39f4e943ed7087b5aa8b350a2ba"
server_key_version: "encryptionkey@0"
I20260810 07:20:28.882850 13213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "31e1c407fdca451f95315081115ce9c1"
format_stamp: "Formatted at 2026-08-10 07:20:28 on dist-test-slave-44d2"
server_key: "3bb345041e700c86c77b944d07afea74"
server_key_iv: "71eac39f4e943ed7087b5aa8b350a2ba"
server_key_version: "encryptionkey@0"
I20260810 07:20:28.886685 13213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 07:20:28.889199 13232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:28.890362 13213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:28.890516 13213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "31e1c407fdca451f95315081115ce9c1"
format_stamp: "Formatted at 2026-08-10 07:20:28 on dist-test-slave-44d2"
server_key: "3bb345041e700c86c77b944d07afea74"
server_key_iv: "71eac39f4e943ed7087b5aa8b350a2ba"
server_key_version: "encryptionkey@0"
I20260810 07:20:28.890643 13213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:28.913962 13213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:28.917467 13213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:28.917696 13213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:28.919236 13213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:28.919312 13213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:28.919385 13213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:28.919448 13213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:20:28 dist-test-slave-44d2 krb5kdc[12841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346428, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:28.931048 13213 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:43403
I20260810 07:20:28.932261 13213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:28.932673 13392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:43403 every 8 connection(s)
I20260810 07:20:28.934325 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:28.919999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:37093 (local address 127.1.125.254:34167)
0810 07:20:28.920134 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:28.920137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:28.921281 (+ 1144us) server_negotiation.cc:407] Connection header received
0810 07:20:28.922293 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:28.922297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:28.922358 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:28.922448 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:28.924243 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.924900 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.926419 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.926659 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.926777 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:28.930112 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:28.930115 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:28.930118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:28.930147 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:28.931950 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:28.932700 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:28.932704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:28.932706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:28.932805 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:28.933194 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:28.933197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:28.933199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:28.933390 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:28.933506 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:28.934031 (+ 525us) server_negotiation.cc:299] Negotiation successful
0810 07:20:28.934153 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:20:28.935036 13245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:28.920585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34167 (local address 127.1.125.194:37093)
0810 07:20:28.921128 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:28.921172 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:28.922067 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:28.922778 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:28.922789 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:28.923334 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:28.924041 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:28.924079 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.925071 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.925081 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:28.926293 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:28.926303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.926840 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.926844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:28.927035 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:28.927645 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:28.927661 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:28.929937 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:28.932083 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:28.932090 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:28.932104 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:28.932578 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:28.932898 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:28.932902 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:28.932904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:28.933094 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:28.933515 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:28.933525 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:28.933756 (+ 231us) client_negotiation.cc:769] Sending connection context
0810 07:20:28.933942 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 07:20:28.934260 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":128,"threads_started":1}
I20260810 07:20:28.936177 13393 heartbeater.cc:347] Connected to a master server at 127.1.125.254:34167
I20260810 07:20:28.936348 13393 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:28.936866 13393 heartbeater.cc:511] Master 127.1.125.254:34167 requested a full tablet report, sending...
I20260810 07:20:28.937963 12905 ts_manager.cc:194] Registered new tserver with Master: 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403)
I20260810 07:20:28.938705 12905 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:37093
I20260810 07:20:28.942560 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 13213
I20260810 07:20:28.942704 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:28.943066 1527 external_mini_cluster.cc:1428] Setting key 11996f2e345a26aced51be672d85c05e
I20260810 07:20:28.946072 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:28.938564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:42169 (local address 127.1.125.254:34167)
0810 07:20:28.938681 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:28.938684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:28.939120 (+ 436us) server_negotiation.cc:407] Connection header received
0810 07:20:28.939272 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:28.939275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:28.939316 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:28.939401 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:28.939838 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.940331 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.941219 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.941447 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.941562 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:28.942699 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:28.942702 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:28.942705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:28.942751 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:28.944390 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:28.944837 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:28.944841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:28.944843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:28.944932 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:28.945221 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:28.945225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:28.945227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:28.945440 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:28.945550 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:28.945808 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 07:20:28.945916 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:20:28.946082 13401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:28.938476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34167 (local address 127.1.125.194:42169)
0810 07:20:28.939028 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:28.939040 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:28.939165 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:28.939419 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:28.939421 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:28.939625 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:28.939723 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:28.939729 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.940462 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.940465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:28.941097 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:28.941106 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:28.941593 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:28.941597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:28.941693 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:28.942118 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:28.942129 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:28.942512 (+ 383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:28.944521 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:28.944527 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:28.944530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:28.944733 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:28.945036 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:28.945039 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:28.945041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:28.945126 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:28.945541 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:28.945544 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:28.945645 (+ 101us) client_negotiation.cc:769] Sending connection context
0810 07:20:28.945754 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 07:20:28.945904 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for oryx/127.1.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab.
Entry for principal oryx/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab.
I20260810 07:20:28.972055 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:34167
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab
--principal=oryx/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:45797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:29.097465 13406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:29.097759 13406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:29.097833 13406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:29.102015 13406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:29.102105 13406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:29.102178 13406 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:20:29.102258 13406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:29.107596 13406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab
--principal=oryx/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.125.254:34167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:29.108803 13406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:29.109839 13406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:29.119729 13414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:29.119741 13413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:29.119741 13421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:29.120042 13406 server_base.cc:1034] running on GCE node
I20260810 07:20:29.120522 13406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:29.121186 13406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:29.122373 13406 hybrid_clock.cc:648] HybridClock initialized: now 1786346429122351 us; error 31 us; skew 500 ppm
Aug 10 07:20:29 dist-test-slave-44d2 krb5kdc[12841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346429, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:29.125218 13406 init.cc:377] Logged in from keytab as oryx/127.1.125.195@KRBTEST.COM (short username oryx)
I20260810 07:20:29.126065 13406 webserver.cc:460] Webserver started at http://127.1.125.195:43143/ using document root <none> and password file <none>
I20260810 07:20:29.126632 13406 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:29.126724 13406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:29.126959 13406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:29.129067 13406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "117d2c7dbd9543e68149dc64fc6ba6a2"
format_stamp: "Formatted at 2026-08-10 07:20:29 on dist-test-slave-44d2"
server_key: "583a8e487a505b3575da77953a10ab31"
server_key_iv: "fb4c1b42630d6c23ad918333d9d250dc"
server_key_version: "encryptionkey@0"
I20260810 07:20:29.129652 13406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "117d2c7dbd9543e68149dc64fc6ba6a2"
format_stamp: "Formatted at 2026-08-10 07:20:29 on dist-test-slave-44d2"
server_key: "583a8e487a505b3575da77953a10ab31"
server_key_iv: "fb4c1b42630d6c23ad918333d9d250dc"
server_key_version: "encryptionkey@0"
I20260810 07:20:29.133602 13406 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 07:20:29.135967 13429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:29.137475 13406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:29.137629 13406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "117d2c7dbd9543e68149dc64fc6ba6a2"
format_stamp: "Formatted at 2026-08-10 07:20:29 on dist-test-slave-44d2"
server_key: "583a8e487a505b3575da77953a10ab31"
server_key_iv: "fb4c1b42630d6c23ad918333d9d250dc"
server_key_version: "encryptionkey@0"
I20260810 07:20:29.137740 13406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:29.160591 13406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:29.163323 13406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:29.163532 13406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:29.165086 13406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:29.165135 13406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:29.165174 13406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:29.165225 13406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:20:29 dist-test-slave-44d2 krb5kdc[12841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346429, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:29.177389 13406 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:39575
I20260810 07:20:29.177472 13586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:39575 every 8 connection(s)
I20260810 07:20:29.178292 13406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:29.179448 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 13406
I20260810 07:20:29.179608 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:29.179966 1527 external_mini_cluster.cc:1428] Setting key 7210a462507a711f5ff05dbf103a811b
I20260810 07:20:29.180722 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.165817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:49149 (local address 127.1.125.254:34167)
0810 07:20:29.165976 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:29.165981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:29.166931 (+ 950us) server_negotiation.cc:407] Connection header received
0810 07:20:29.167898 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:29.167901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:29.167984 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:29.168084 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:29.169590 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.170390 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.171936 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.172188 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.172313 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.176285 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:29.176289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:29.176292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:29.176322 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:29.178070 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:29.178689 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:29.178692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:29.178694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:29.178803 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:29.179241 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:29.179245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:29.179246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:29.179505 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:29.179646 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:29.180353 (+ 707us) server_negotiation.cc:299] Negotiation successful
0810 07:20:29.180525 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 07:20:29.181702 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.166257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34167 (local address 127.1.125.195:49149)
0810 07:20:29.166755 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:29.166806 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:29.167679 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:29.168253 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:29.168264 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:29.168767 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:29.169437 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.169453 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.170525 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.170529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.171802 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.171813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.172380 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.172385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.172638 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.173299 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:29.173326 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:29.176068 (+ 2742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:29.178207 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:29.178213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:29.178228 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:29.178565 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:29.178919 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:29.178922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:29.178924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:29.179114 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:29.179713 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:29.179721 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:29.180032 (+ 311us) client_negotiation.cc:769] Sending connection context
0810 07:20:29.180266 (+ 234us) client_negotiation.cc:240] Negotiation successful
0810 07:20:29.180653 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":139,"threads_started":1}
I20260810 07:20:29.183090 13589 heartbeater.cc:347] Connected to a master server at 127.1.125.254:34167
I20260810 07:20:29.183239 13589 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:29.183826 13589 heartbeater.cc:511] Master 127.1.125.254:34167 requested a full tablet report, sending...
I20260810 07:20:29.184955 12904 ts_manager.cc:194] Registered new tserver with Master: 117d2c7dbd9543e68149dc64fc6ba6a2 (127.1.125.195:39575)
I20260810 07:20:29.185693 12904 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:49149
I20260810 07:20:29.192203 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.185447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:54083 (local address 127.1.125.254:34167)
0810 07:20:29.185596 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:29.185600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:29.185827 (+ 227us) server_negotiation.cc:407] Connection header received
0810 07:20:29.185936 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:29.185938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:29.185977 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:29.186070 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:29.186592 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.187075 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.188071 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.188276 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.188356 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.189386 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:29.189389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:29.189391 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:29.189412 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:29.190905 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:29.191270 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:29.191272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:29.191274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:29.191344 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:29.191576 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:29.191578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:29.191579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:29.191730 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:29.191804 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:29.192013 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 07:20:29.192097 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:20:29.192217 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.185381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34167 (local address 127.1.125.195:54083)
0810 07:20:29.185727 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:29.185742 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:29.185845 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:29.186091 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:29.186094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:29.186320 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:29.186469 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.186478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.187207 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.187211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.187964 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.187974 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.188374 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.188377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.188443 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.188936 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:29.188948 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:29.189266 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:29.191000 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:29.191002 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:29.191004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:29.191188 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:29.191424 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:29.191426 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:29.191428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:29.191501 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:29.191811 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:29.191813 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:29.191887 (+ 74us) client_negotiation.cc:769] Sending connection context
0810 07:20:29.191965 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:20:29.192069 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":4,"thread_start_us":159,"threads_started":1}
I20260810 07:20:29.194093 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:20:29 dist-test-slave-44d2 krb5kdc[12841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346429, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:20:29 dist-test-slave-44d2 krb5kdc[12841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:29 dist-test-slave-44d2 krb5kdc[12841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:20:29.220535 12976 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:29.214162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37716 (local address 127.1.125.254:34167)
0810 07:20:29.214302 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:29.214306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:29.214318 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:29.214358 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:29.214360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:29.214398 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:29.214494 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:29.215220 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.215751 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.216668 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.216887 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.217055 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.220356 (+ 3301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37716: BlockingRecv error: recv got EOF from 127.0.0.1:37716 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 07:20:29 dist-test-slave-44d2 krb5kdc[12841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346429, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:29.231058 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.222623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37726 (local address 127.1.125.254:34167)
0810 07:20:29.222776 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:29.222781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:29.222961 (+ 180us) server_negotiation.cc:407] Connection header received
0810 07:20:29.223122 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:29.223125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:29.223171 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:29.223259 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:29.223875 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.224386 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.225229 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.225447 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.225561 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.227865 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:29.227868 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:29.227870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:29.227892 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:29.229285 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:29.229848 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:29.229851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:29.229852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:29.229927 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:29.230185 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:29.230187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:29.230188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:29.230347 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:29.230449 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:29.230744 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 07:20:29.230864 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 07:20:29.234401 12904 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:20:29.238327 12904 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:29.252764 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.247726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44398 (local address 127.1.125.195:39575)
0810 07:20:29.248112 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:29.248125 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:29.248463 (+ 338us) server_negotiation.cc:407] Connection header received
0810 07:20:29.248576 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:29.248584 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:29.248772 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:29.248926 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:29.249127 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.250202 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.251414 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.252015 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.252142 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.252334 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:29.252472 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 07:20:29.252629 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":130,"threads_started":1}
I20260810 07:20:29.252941 13628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.246535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:43403 (local address 127.0.0.1:37182)
0810 07:20:29.246900 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:29.246953 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:29.247089 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:29.248027 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:29.248034 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:29.248059 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:29.248208 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.248217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.249676 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.249682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.251075 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.251094 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.252216 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.252220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.252321 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.252361 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 07:20:29.252410 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 07:20:29.252814 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":258,"thread_start_us":96,"threads_started":1}
I20260810 07:20:29.252760 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.247281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:39575 (local address 127.0.0.1:44398)
0810 07:20:29.248247 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:29.248261 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:29.248369 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:29.248910 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:29.248913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:29.248933 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:29.249012 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.249018 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.250310 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.250314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.251271 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.251280 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.252151 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.252154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.252271 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.252318 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 07:20:29.252439 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 07:20:29.252599 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"mutex_wait_us":28,"thread_start_us":94,"threads_started":1}
I20260810 07:20:29.253123 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.246732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37182 (local address 127.1.125.194:43403)
0810 07:20:29.247452 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:29.247466 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:29.247494 (+ 28us) server_negotiation.cc:407] Connection header received
0810 07:20:29.247595 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:29.247600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:29.247813 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:29.248380 (+ 567us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:29.248562 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.249550 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.251223 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.252053 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.252643 (+ 590us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.252818 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:29.252869 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 07:20:29.252956 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":104,"threads_started":1}
I20260810 07:20:29.254767 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.246956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:37879 (local address 127.0.0.1:50934)
0810 07:20:29.247254 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:29.247267 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:29.247357 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:29.248389 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:29.248394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:29.248415 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:29.248548 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.248555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.250112 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.250116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.251336 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.251344 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.252378 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.252383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.252465 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.252615 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 07:20:29.252666 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 07:20:29.252730 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":190,"thread_start_us":129,"threads_started":1}
I20260810 07:20:29.255594 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.247431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50934 (local address 127.1.125.193:37879)
0810 07:20:29.247747 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:29.247754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:29.247780 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:20:29.247866 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:29.247873 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:29.248099 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:29.248692 (+ 593us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:29.248840 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.249970 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.251474 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.252256 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.255139 (+ 2883us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.255299 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:29.255355 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 07:20:29.255449 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":103,"threads_started":1}
I20260810 07:20:29.257336 13501 tablet_service.cc:1467] Processing CreateTablet for tablet b8b3368771e84c7e9315c381d2b0b357 (DEFAULT_TABLE table=test-table [id=49aa77b299e14e24b6701cded3dd4eaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:29.257339 13304 tablet_service.cc:1467] Processing CreateTablet for tablet b8b3368771e84c7e9315c381d2b0b357 (DEFAULT_TABLE table=test-table [id=49aa77b299e14e24b6701cded3dd4eaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:29.258020 13109 tablet_service.cc:1467] Processing CreateTablet for tablet b8b3368771e84c7e9315c381d2b0b357 (DEFAULT_TABLE table=test-table [id=49aa77b299e14e24b6701cded3dd4eaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:29.258492 13304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8b3368771e84c7e9315c381d2b0b357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:29.258548 13501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8b3368771e84c7e9315c381d2b0b357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:29.258966 13109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8b3368771e84c7e9315c381d2b0b357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:29.265101 13641 tablet_bootstrap.cc:492] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2: Bootstrap starting.
I20260810 07:20:29.266601 13643 tablet_bootstrap.cc:492] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1: Bootstrap starting.
I20260810 07:20:29.267163 13642 tablet_bootstrap.cc:492] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99: Bootstrap starting.
I20260810 07:20:29.267717 13641 tablet_bootstrap.cc:654] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:29.268702 13641 log.cc:824] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:29.269040 13643 tablet_bootstrap.cc:654] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:29.269105 13642 tablet_bootstrap.cc:654] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:29.269929 13642 log.cc:824] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:29.269938 13643 log.cc:824] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:29.270826 13641 tablet_bootstrap.cc:492] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2: No bootstrap required, opened a new log
I20260810 07:20:29.271046 13641 ts_tablet_manager.cc:1397] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260810 07:20:29.274302 13643 tablet_bootstrap.cc:492] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1: No bootstrap required, opened a new log
I20260810 07:20:29.274531 13643 ts_tablet_manager.cc:1397] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260810 07:20:29.274663 13642 tablet_bootstrap.cc:492] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99: No bootstrap required, opened a new log
I20260810 07:20:29.274933 13642 ts_tablet_manager.cc:1397] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 07:20:29.274876 13641 raft_consensus.cc:348] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } }
I20260810 07:20:29.275144 13641 raft_consensus.cc:374] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:29.275202 13641 raft_consensus.cc:729] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d2c7dbd9543e68149dc64fc6ba6a2, State: Initialized, Role: FOLLOWER
I20260810 07:20:29.275663 13641 consensus_queue.cc:260] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } }
I20260810 07:20:29.276379 13589 heartbeater.cc:503] Master 127.1.125.254:34167 was elected leader, sending a full tablet report...
I20260810 07:20:29.276866 13641 ts_tablet_manager.cc:1428] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260810 07:20:29.278667 13643 raft_consensus.cc:348] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } }
I20260810 07:20:29.278931 13643 raft_consensus.cc:374] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:29.278992 13643 raft_consensus.cc:729] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e1c407fdca451f95315081115ce9c1, State: Initialized, Role: FOLLOWER
I20260810 07:20:29.279444 13643 consensus_queue.cc:260] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } }
I20260810 07:20:29.279417 13642 raft_consensus.cc:348] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } }
I20260810 07:20:29.279680 13642 raft_consensus.cc:374] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:29.279750 13642 raft_consensus.cc:729] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6901a7f9e3e4f4d902b29bc7a63dc99, State: Initialized, Role: FOLLOWER
I20260810 07:20:29.280184 13642 consensus_queue.cc:260] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } }
I20260810 07:20:29.280469 13643 ts_tablet_manager.cc:1428] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 07:20:29.280838 13393 heartbeater.cc:503] Master 127.1.125.254:34167 was elected leader, sending a full tablet report...
I20260810 07:20:29.281136 13642 ts_tablet_manager.cc:1428] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260810 07:20:29.281591 13202 heartbeater.cc:503] Master 127.1.125.254:34167 was elected leader, sending a full tablet report...
W20260810 07:20:29.429389 13591 tablet.cc:2369] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:20:29.433585 13394 tablet.cc:2369] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:20:29.453884 13203 tablet.cc:2369] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:20:29.632241 13652 raft_consensus.cc:482] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:29.632560 13652 raft_consensus.cc:504] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } }
I20260810 07:20:29.633816 13652 leader_election.cc:288] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403), 117d2c7dbd9543e68149dc64fc6ba6a2 (127.1.125.195:39575)
I20260810 07:20:29.638172 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.634181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:47075 (local address 127.1.125.194:43403)
0810 07:20:29.634325 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:29.634329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:29.634543 (+ 214us) server_negotiation.cc:407] Connection header received
0810 07:20:29.634756 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:29.634760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:29.634816 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:29.634881 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:29.635252 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.635926 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.637207 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.637610 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.637744 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.637798 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:29.637977 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 07:20:29.638050 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:20:29.638201 13653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.634107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:43403 (local address 127.1.125.193:47075)
0810 07:20:29.634423 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:29.634440 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:29.634569 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:29.635017 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:29.635019 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:29.635036 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:29.635137 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.635142 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.636041 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.636044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.637077 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.637090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.637737 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.637741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.637842 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.637868 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:20:29.637960 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 07:20:29.638032 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":26,"thread_start_us":145,"threads_started":1}
I20260810 07:20:29.638428 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.634776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:60475 (local address 127.1.125.195:39575)
0810 07:20:29.634912 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:29.634917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:29.635052 (+ 135us) server_negotiation.cc:407] Connection header received
0810 07:20:29.635218 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:29.635227 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:29.635269 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:29.635350 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:29.635666 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.636158 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.637491 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.637877 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.638018 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.638067 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:29.638240 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 07:20:29.638308 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 07:20:29.638443 13654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.634594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:39575 (local address 127.1.125.193:60475)
0810 07:20:29.634925 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:29.634940 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:29.635101 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:29.635377 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:29.635380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:29.635396 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:29.635527 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.635533 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.636273 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.636278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.637328 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.637339 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.638004 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.638008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.638108 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.638130 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:20:29.638224 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 07:20:29.638290 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
I20260810 07:20:29.638916 13332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b3368771e84c7e9315c381d2b0b357" candidate_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e1c407fdca451f95315081115ce9c1" is_pre_election: true
I20260810 07:20:29.639261 13332 raft_consensus.cc:2454] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6901a7f9e3e4f4d902b29bc7a63dc99 in term 0.
I20260810 07:20:29.639432 13528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b3368771e84c7e9315c381d2b0b357" candidate_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" is_pre_election: true
I20260810 07:20:29.639746 13528 raft_consensus.cc:2454] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6901a7f9e3e4f4d902b29bc7a63dc99 in term 0.
I20260810 07:20:29.639855 13037 leader_election.cc:302] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31e1c407fdca451f95315081115ce9c1, e6901a7f9e3e4f4d902b29bc7a63dc99; no voters:
I20260810 07:20:29.640170 13652 raft_consensus.cc:2781] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:29.640249 13652 raft_consensus.cc:482] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:29.640280 13652 raft_consensus.cc:3037] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:29.641556 13652 raft_consensus.cc:504] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } }
I20260810 07:20:29.641897 13652 leader_election.cc:288] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [CANDIDATE]: Term 1 election: Requested vote from peers 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403), 117d2c7dbd9543e68149dc64fc6ba6a2 (127.1.125.195:39575)
I20260810 07:20:29.642329 13528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b3368771e84c7e9315c381d2b0b357" candidate_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2"
I20260810 07:20:29.642510 13528 raft_consensus.cc:3037] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:29.643587 13528 raft_consensus.cc:2454] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6901a7f9e3e4f4d902b29bc7a63dc99 in term 1.
I20260810 07:20:29.643810 13332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b3368771e84c7e9315c381d2b0b357" candidate_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e1c407fdca451f95315081115ce9c1"
I20260810 07:20:29.643893 13037 leader_election.cc:302] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d2c7dbd9543e68149dc64fc6ba6a2, e6901a7f9e3e4f4d902b29bc7a63dc99; no voters:
I20260810 07:20:29.643941 13332 raft_consensus.cc:3037] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:29.644104 13652 raft_consensus.cc:2781] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:29.644327 13652 raft_consensus.cc:686] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 LEADER]: Becoming Leader. State: Replica: e6901a7f9e3e4f4d902b29bc7a63dc99, State: Running, Role: LEADER
I20260810 07:20:29.644695 13652 consensus_queue.cc:237] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } }
I20260810 07:20:29.645084 13332 raft_consensus.cc:2454] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6901a7f9e3e4f4d902b29bc7a63dc99 in term 1.
I20260810 07:20:29.647660 12905 catalog_manager.cc:5495] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6901a7f9e3e4f4d902b29bc7a63dc99 (127.1.125.193). New cstate: current_term: 1 leader_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } health_report { overall_health: HEALTHY } } }
I20260810 07:20:29.714107 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.710326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50938 (local address 127.1.125.193:37879)
0810 07:20:29.710458 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:29.710463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:29.710486 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:20:29.710655 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:29.710658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:29.710721 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:29.710831 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:29.711102 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.711628 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.712620 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.712855 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.712970 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.713278 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:29.713698 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:29.713830 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:29.713891 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 07:20:29.713954 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:20:29.719919 13528 raft_consensus.cc:1261] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Refusing update from remote peer e6901a7f9e3e4f4d902b29bc7a63dc99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:29.719918 13332 raft_consensus.cc:1261] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Refusing update from remote peer e6901a7f9e3e4f4d902b29bc7a63dc99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:29.720649 13652 consensus_queue.cc:1035] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:29.720886 13655 consensus_queue.cc:1035] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:29.728961 13662 mvcc.cc:205] Tried to move back new op lower bound from 7316874976125722624 to 7316874975827398656. Current Snapshot: MvccSnapshot[applied={T|T < 7316874976125722624}]
I20260810 07:20:29.736330 12904 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:29.736544 12904 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:29.739094 12904 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d: Sending DeleteTablet for 3 replicas of tablet b8b3368771e84c7e9315c381d2b0b357
I20260810 07:20:29.739921 13109 tablet_service.cc:1514] Processing DeleteTablet for tablet b8b3368771e84c7e9315c381d2b0b357 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:29 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:50934
I20260810 07:20:29.739936 13501 tablet_service.cc:1514] Processing DeleteTablet for tablet b8b3368771e84c7e9315c381d2b0b357 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:29 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:44398
I20260810 07:20:29.740393 13677 tablet_replica.cc:330] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2: stopping tablet replica
I20260810 07:20:29.740532 12904 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:20:29.740741 13677 raft_consensus.cc:2229] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:29.740715 13304 tablet_service.cc:1514] Processing DeleteTablet for tablet b8b3368771e84c7e9315c381d2b0b357 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:29 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:37182
W20260810 07:20:29.740908 12904 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:29.741029 13677 raft_consensus.cc:2258] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:29.741573 13678 tablet_replica.cc:330] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1: stopping tablet replica
I20260810 07:20:29.741940 13678 raft_consensus.cc:2229] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:29.742353 13678 raft_consensus.cc:2258] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:29.742590 13677 ts_tablet_manager.cc:1905] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:29.744132 13675 tablet_replica.cc:330] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99: stopping tablet replica
I20260810 07:20:29.745440 13678 ts_tablet_manager.cc:1905] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:29.745517 13677 ts_tablet_manager.cc:1918] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:29.745635 13677 log.cc:1196] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/b8b3368771e84c7e9315c381d2b0b357
I20260810 07:20:29.746027 13677 ts_tablet_manager.cc:1939] T b8b3368771e84c7e9315c381d2b0b357 P 117d2c7dbd9543e68149dc64fc6ba6a2: Deleting consensus metadata
I20260810 07:20:29.746673 13304 tablet_service.cc:1467] Processing CreateTablet for tablet cddb0d5e2b264417a5dc6304c171fd33 (DEFAULT_TABLE table=test-table [id=4eaa18bbf18745348de0507fa6949c0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:29.746938 13304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cddb0d5e2b264417a5dc6304c171fd33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:29.744403 13675 raft_consensus.cc:2229] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:29.746412 13109 tablet_service.cc:1467] Processing CreateTablet for tablet cddb0d5e2b264417a5dc6304c171fd33 (DEFAULT_TABLE table=test-table [id=4eaa18bbf18745348de0507fa6949c0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:29.747372 13109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cddb0d5e2b264417a5dc6304c171fd33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:29.747473 13675 raft_consensus.cc:2258] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:29.749341 13643 tablet_bootstrap.cc:492] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1: Bootstrap starting.
I20260810 07:20:29.749462 13501 tablet_service.cc:1467] Processing CreateTablet for tablet cddb0d5e2b264417a5dc6304c171fd33 (DEFAULT_TABLE table=test-table [id=4eaa18bbf18745348de0507fa6949c0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:29.749696 13501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cddb0d5e2b264417a5dc6304c171fd33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:29.750252 12886 catalog_manager.cc:4841] TS 117d2c7dbd9543e68149dc64fc6ba6a2 (127.1.125.195:39575): tablet b8b3368771e84c7e9315c381d2b0b357 (table test-table [id=49aa77b299e14e24b6701cded3dd4eaa]) successfully deleted
I20260810 07:20:29.750349 13643 tablet_bootstrap.cc:654] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:29.750972 13642 tablet_bootstrap.cc:492] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99: Bootstrap starting.
I20260810 07:20:29.751621 13643 tablet_bootstrap.cc:492] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1: No bootstrap required, opened a new log
I20260810 07:20:29.751695 13643 ts_tablet_manager.cc:1397] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 07:20:29.752084 13642 tablet_bootstrap.cc:654] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:29.752171 13641 tablet_bootstrap.cc:492] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2: Bootstrap starting.
I20260810 07:20:29.752556 13643 raft_consensus.cc:348] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.752698 13643 raft_consensus.cc:374] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:29.752740 13643 raft_consensus.cc:729] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e1c407fdca451f95315081115ce9c1, State: Initialized, Role: FOLLOWER
I20260810 07:20:29.752933 13643 consensus_queue.cc:260] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.753141 13641 tablet_bootstrap.cc:654] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:29.753763 13643 ts_tablet_manager.cc:1428] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260810 07:20:29.754423 13675 ts_tablet_manager.cc:1905] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:29.755100 13641 tablet_bootstrap.cc:492] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2: No bootstrap required, opened a new log
I20260810 07:20:29.755185 13641 ts_tablet_manager.cc:1397] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:20:29.755607 13641 raft_consensus.cc:348] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.755703 13641 raft_consensus.cc:374] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:29.755733 13641 raft_consensus.cc:729] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d2c7dbd9543e68149dc64fc6ba6a2, State: Initialized, Role: FOLLOWER
I20260810 07:20:29.755995 13641 consensus_queue.cc:260] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.757249 13642 tablet_bootstrap.cc:492] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99: No bootstrap required, opened a new log
I20260810 07:20:29.757316 13642 ts_tablet_manager.cc:1397] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260810 07:20:29.757692 13641 ts_tablet_manager.cc:1428] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:20:29.757762 13642 raft_consensus.cc:348] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.757879 13642 raft_consensus.cc:374] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:29.757925 13642 raft_consensus.cc:729] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6901a7f9e3e4f4d902b29bc7a63dc99, State: Initialized, Role: FOLLOWER
I20260810 07:20:29.758056 13642 consensus_queue.cc:260] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.758339 13642 ts_tablet_manager.cc:1428] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:29.760812 13675 ts_tablet_manager.cc:1918] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:29.760927 13675 log.cc:1196] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/wal/wals/b8b3368771e84c7e9315c381d2b0b357
I20260810 07:20:29.761258 13678 ts_tablet_manager.cc:1918] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:29.761323 13675 ts_tablet_manager.cc:1939] T b8b3368771e84c7e9315c381d2b0b357 P e6901a7f9e3e4f4d902b29bc7a63dc99: Deleting consensus metadata
I20260810 07:20:29.761446 13678 log.cc:1196] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/b8b3368771e84c7e9315c381d2b0b357
I20260810 07:20:29.761777 13678 ts_tablet_manager.cc:1939] T b8b3368771e84c7e9315c381d2b0b357 P 31e1c407fdca451f95315081115ce9c1: Deleting consensus metadata
I20260810 07:20:29.762444 12887 catalog_manager.cc:4841] TS e6901a7f9e3e4f4d902b29bc7a63dc99 (127.1.125.193:37879): tablet b8b3368771e84c7e9315c381d2b0b357 (table test-table [id=49aa77b299e14e24b6701cded3dd4eaa]) successfully deleted
I20260810 07:20:29.762866 12886 catalog_manager.cc:4841] TS 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403): tablet b8b3368771e84c7e9315c381d2b0b357 (table test-table [id=49aa77b299e14e24b6701cded3dd4eaa]) successfully deleted
I20260810 07:20:29.887125 13670 raft_consensus.cc:482] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:29.887264 13670 raft_consensus.cc:504] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.887743 13670 leader_election.cc:288] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403), 117d2c7dbd9543e68149dc64fc6ba6a2 (127.1.125.195:39575)
I20260810 07:20:29.888358 13332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cddb0d5e2b264417a5dc6304c171fd33" candidate_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e1c407fdca451f95315081115ce9c1" is_pre_election: true
I20260810 07:20:29.888412 13528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cddb0d5e2b264417a5dc6304c171fd33" candidate_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" is_pre_election: true
I20260810 07:20:29.888554 13332 raft_consensus.cc:2454] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6901a7f9e3e4f4d902b29bc7a63dc99 in term 0.
I20260810 07:20:29.888671 13528 raft_consensus.cc:2454] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6901a7f9e3e4f4d902b29bc7a63dc99 in term 0.
I20260810 07:20:29.889065 13037 leader_election.cc:302] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d2c7dbd9543e68149dc64fc6ba6a2, e6901a7f9e3e4f4d902b29bc7a63dc99; no voters:
I20260810 07:20:29.889389 13670 raft_consensus.cc:2781] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:29.889439 13670 raft_consensus.cc:482] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:29.889465 13670 raft_consensus.cc:3037] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:29.890499 13670 raft_consensus.cc:504] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.890836 13670 leader_election.cc:288] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [CANDIDATE]: Term 1 election: Requested vote from peers 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403), 117d2c7dbd9543e68149dc64fc6ba6a2 (127.1.125.195:39575)
I20260810 07:20:29.891360 13332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cddb0d5e2b264417a5dc6304c171fd33" candidate_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e1c407fdca451f95315081115ce9c1"
I20260810 07:20:29.891505 13332 raft_consensus.cc:3037] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:29.892820 13528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cddb0d5e2b264417a5dc6304c171fd33" candidate_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2"
I20260810 07:20:29.892973 13332 raft_consensus.cc:2454] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6901a7f9e3e4f4d902b29bc7a63dc99 in term 1.
I20260810 07:20:29.892989 13528 raft_consensus.cc:3037] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:29.893337 13037 leader_election.cc:302] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31e1c407fdca451f95315081115ce9c1, e6901a7f9e3e4f4d902b29bc7a63dc99; no voters:
I20260810 07:20:29.893532 13670 raft_consensus.cc:2781] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:29.893656 13670 raft_consensus.cc:686] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 LEADER]: Becoming Leader. State: Replica: e6901a7f9e3e4f4d902b29bc7a63dc99, State: Running, Role: LEADER
I20260810 07:20:29.893821 13670 consensus_queue.cc:237] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.894487 13528 raft_consensus.cc:2454] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6901a7f9e3e4f4d902b29bc7a63dc99 in term 1.
I20260810 07:20:29.895800 12905 catalog_manager.cc:5495] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6901a7f9e3e4f4d902b29bc7a63dc99 (127.1.125.193). New cstate: current_term: 1 leader_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } health_report { overall_health: UNKNOWN } } }
Aug 10 07:20:29 dist-test-slave-44d2 krb5kdc[12841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346428, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.254@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:29.944738 13689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.934385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34167 (local address 127.0.0.1:37728)
0810 07:20:29.934823 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:29.934841 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:29.934970 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:29.935254 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:29.935257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:29.935750 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:29.935883 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.935891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.936907 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.936910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.937862 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:29.937885 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.938354 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.938358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:29.938458 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.939088 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:29.939102 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:29.940795 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:29.942826 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:29.942837 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:29.942845 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:29.943209 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:29.943492 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:29.943496 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:29.943498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:29.943604 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:29.944025 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:29.944033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:29.944170 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 07:20:29.944294 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 07:20:29.944527 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":149,"threads_started":1}
I20260810 07:20:29.944741 13690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:29.934451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37728 (local address 127.1.125.254:34167)
0810 07:20:29.934826 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:29.934835 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:29.934935 (+ 100us) server_negotiation.cc:407] Connection header received
0810 07:20:29.935066 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:29.935069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:29.935114 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:29.935206 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:29.936012 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.936789 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.937999 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:29.938237 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:29.938328 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:29.940928 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:29.940931 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:29.940933 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:29.940956 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:29.942691 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:29.943315 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:29.943318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:29.943319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:29.943397 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:29.943737 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:29.943740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:29.943741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:29.943919 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:29.943996 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:29.944346 (+ 350us) server_negotiation.cc:299] Negotiation successful
0810 07:20:29.944612 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":300,"thread_start_us":134,"threads_started":1}
I20260810 07:20:29.949676 12905 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:37728:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 07:20:29.950498 12905 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:29.955997 13304 tablet_service.cc:1467] Processing CreateTablet for tablet f5acd9bbd3944bc2ab7c3bdbe23da681 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=652187b7ba574eaab772394b2af9b9d9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:20:29.955997 13109 tablet_service.cc:1467] Processing CreateTablet for tablet f5acd9bbd3944bc2ab7c3bdbe23da681 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=652187b7ba574eaab772394b2af9b9d9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:20:29.956130 13501 tablet_service.cc:1467] Processing CreateTablet for tablet f5acd9bbd3944bc2ab7c3bdbe23da681 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=652187b7ba574eaab772394b2af9b9d9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:20:29.956463 13304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5acd9bbd3944bc2ab7c3bdbe23da681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:29.956463 13109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5acd9bbd3944bc2ab7c3bdbe23da681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:29.956640 13501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5acd9bbd3944bc2ab7c3bdbe23da681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:29.959203 13641 tablet_bootstrap.cc:492] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2: Bootstrap starting.
I20260810 07:20:29.959596 13642 tablet_bootstrap.cc:492] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99: Bootstrap starting.
I20260810 07:20:29.960261 13641 tablet_bootstrap.cc:654] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:29.960443 13642 tablet_bootstrap.cc:654] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:29.961004 13643 tablet_bootstrap.cc:492] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1: Bootstrap starting.
I20260810 07:20:29.961692 13642 tablet_bootstrap.cc:492] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99: No bootstrap required, opened a new log
I20260810 07:20:29.961791 13643 tablet_bootstrap.cc:654] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:29.961777 13642 ts_tablet_manager.cc:1397] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:20:29.962218 13642 raft_consensus.cc:348] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.962368 13642 raft_consensus.cc:374] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:29.962421 13642 raft_consensus.cc:729] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6901a7f9e3e4f4d902b29bc7a63dc99, State: Initialized, Role: FOLLOWER
I20260810 07:20:29.962584 13642 consensus_queue.cc:260] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.962795 13641 tablet_bootstrap.cc:492] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2: No bootstrap required, opened a new log
I20260810 07:20:29.962898 13641 ts_tablet_manager.cc:1397] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 07:20:29.962958 13670 tablet_replica.cc:439] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } }
I20260810 07:20:29.963099 13643 tablet_bootstrap.cc:492] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1: No bootstrap required, opened a new log
I20260810 07:20:29.963127 13670 tablet_replica.cc:442] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:29.963182 13643 ts_tablet_manager.cc:1397] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:20:29.963382 13641 raft_consensus.cc:348] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.963497 13641 raft_consensus.cc:374] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:29.963532 13641 raft_consensus.cc:729] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d2c7dbd9543e68149dc64fc6ba6a2, State: Initialized, Role: FOLLOWER
I20260810 07:20:29.963654 13643 raft_consensus.cc:348] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.963762 13643 raft_consensus.cc:374] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:29.963792 13643 raft_consensus.cc:729] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e1c407fdca451f95315081115ce9c1, State: Initialized, Role: FOLLOWER
I20260810 07:20:29.963913 13643 consensus_queue.cc:260] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.963979 13642 ts_tablet_manager.cc:1428] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:29.964201 13643 ts_tablet_manager.cc:1428] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:29.964201 13641 consensus_queue.cc:260] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:29.964169 13650 tablet_replica.cc:439] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } }
I20260810 07:20:29.964339 13650 tablet_replica.cc:442] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:29.964459 13641 ts_tablet_manager.cc:1428] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:20:29.965026 13649 tablet_replica.cc:439] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } }
I20260810 07:20:29.965193 13649 tablet_replica.cc:442] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:30.093811 13670 raft_consensus.cc:482] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:30.094017 13670 raft_consensus.cc:504] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.094441 13670 leader_election.cc:288] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403), 117d2c7dbd9543e68149dc64fc6ba6a2 (127.1.125.195:39575)
I20260810 07:20:30.095059 13332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5acd9bbd3944bc2ab7c3bdbe23da681" candidate_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e1c407fdca451f95315081115ce9c1" is_pre_election: true
I20260810 07:20:30.095062 13528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5acd9bbd3944bc2ab7c3bdbe23da681" candidate_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" is_pre_election: true
I20260810 07:20:30.095232 13332 raft_consensus.cc:2454] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6901a7f9e3e4f4d902b29bc7a63dc99 in term 0.
I20260810 07:20:30.095243 13528 raft_consensus.cc:2454] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6901a7f9e3e4f4d902b29bc7a63dc99 in term 0.
I20260810 07:20:30.095566 13037 leader_election.cc:302] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31e1c407fdca451f95315081115ce9c1, e6901a7f9e3e4f4d902b29bc7a63dc99; no voters:
I20260810 07:20:30.095785 13670 raft_consensus.cc:2781] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:30.095862 13670 raft_consensus.cc:482] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:30.095916 13670 raft_consensus.cc:3037] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:30.097020 13670 raft_consensus.cc:504] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.097350 13670 leader_election.cc:288] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [CANDIDATE]: Term 1 election: Requested vote from peers 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403), 117d2c7dbd9543e68149dc64fc6ba6a2 (127.1.125.195:39575)
I20260810 07:20:30.097738 13332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5acd9bbd3944bc2ab7c3bdbe23da681" candidate_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e1c407fdca451f95315081115ce9c1"
I20260810 07:20:30.097785 13528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5acd9bbd3944bc2ab7c3bdbe23da681" candidate_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2"
I20260810 07:20:30.097886 13332 raft_consensus.cc:3037] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:30.097911 13528 raft_consensus.cc:3037] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:30.098780 13332 raft_consensus.cc:2454] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6901a7f9e3e4f4d902b29bc7a63dc99 in term 1.
I20260810 07:20:30.098870 13528 raft_consensus.cc:2454] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6901a7f9e3e4f4d902b29bc7a63dc99 in term 1.
I20260810 07:20:30.099092 13037 leader_election.cc:302] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31e1c407fdca451f95315081115ce9c1, e6901a7f9e3e4f4d902b29bc7a63dc99; no voters:
I20260810 07:20:30.099267 13670 raft_consensus.cc:2781] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:30.099390 13670 raft_consensus.cc:686] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 LEADER]: Becoming Leader. State: Replica: e6901a7f9e3e4f4d902b29bc7a63dc99, State: Running, Role: LEADER
I20260810 07:20:30.099555 13670 consensus_queue.cc:237] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.100292 13652 tablet_replica.cc:439] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99: TxnStatusTablet state changed. Reason: New leader e6901a7f9e3e4f4d902b29bc7a63dc99. Latest consensus state: current_term: 1 leader_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } }
I20260810 07:20:30.100411 13652 tablet_replica.cc:442] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:20:30.100751 13702 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 07:20:30.100955 13702 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 07:20:30.101481 12903 catalog_manager.cc:5495] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6901a7f9e3e4f4d902b29bc7a63dc99 (127.1.125.193). New cstate: current_term: 1 leader_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } health_report { overall_health: UNKNOWN } } }
I20260810 07:20:30.124011 13689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.120347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:37879 (local address 127.0.0.1:50946)
0810 07:20:30.120546 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:30.120563 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:30.120709 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:30.121015 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:30.121017 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:30.121024 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:20:30.121135 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:30.121140 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.122149 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.122151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:30.122748 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:30.122757 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.123312 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.123315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:30.123394 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.123438 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:20:30.123748 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:20:30.123760 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 07:20:30.123810 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:20:30.123878 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:20:30.124038 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.120511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50946 (local address 127.1.125.193:37879)
0810 07:20:30.120643 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:30.120648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:30.120664 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:20:30.120832 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:30.120836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:30.120893 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:30.120980 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:30.121241 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.122025 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.122860 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.123203 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.123309 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.123569 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:30.123661 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:30.123734 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:30.123847 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 07:20:30.123906 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:20:30.126636 13528 raft_consensus.cc:1261] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Refusing update from remote peer e6901a7f9e3e4f4d902b29bc7a63dc99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:30.126760 13332 raft_consensus.cc:1261] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Refusing update from remote peer e6901a7f9e3e4f4d902b29bc7a63dc99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:30.127095 13652 consensus_queue.cc:1035] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:30.127314 13670 consensus_queue.cc:1035] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:30.128612 13649 tablet_replica.cc:439] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2: TxnStatusTablet state changed. Reason: New leader e6901a7f9e3e4f4d902b29bc7a63dc99. Latest consensus state: current_term: 1 leader_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } }
I20260810 07:20:30.128726 13649 tablet_replica.cc:442] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:30.128827 13650 tablet_replica.cc:439] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1: TxnStatusTablet state changed. Reason: New leader e6901a7f9e3e4f4d902b29bc7a63dc99. Latest consensus state: current_term: 1 leader_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } }
I20260810 07:20:30.128939 13650 tablet_replica.cc:442] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:30.130726 13652 tablet_replica.cc:439] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } }
I20260810 07:20:30.130826 13652 tablet_replica.cc:442] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:20:30.131767 13660 mvcc.cc:205] Tried to move back new op lower bound from 7316874977794330624 to 7316874977689276416. Current Snapshot: MvccSnapshot[applied={T|T < 7316874977794330624 or (T in {7316874977794330624})}]
I20260810 07:20:30.132581 13652 tablet_replica.cc:439] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } }
I20260810 07:20:30.132689 13652 tablet_replica.cc:442] T f5acd9bbd3944bc2ab7c3bdbe23da681 P e6901a7f9e3e4f4d902b29bc7a63dc99: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:20:30.132828 13650 tablet_replica.cc:439] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } }
I20260810 07:20:30.132934 13650 tablet_replica.cc:442] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 31e1c407fdca451f95315081115ce9c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:30.133746 13665 mvcc.cc:205] Tried to move back new op lower bound from 7316874977794330624 to 7316874977689276416. Current Snapshot: MvccSnapshot[applied={T|T < 7316874977794330624}]
I20260810 07:20:30.134352 13649 tablet_replica.cc:439] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } } }
I20260810 07:20:30.134455 13649 tablet_replica.cc:442] T f5acd9bbd3944bc2ab7c3bdbe23da681 P 117d2c7dbd9543e68149dc64fc6ba6a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:20:30.146978 13711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.143510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:37879 (local address 127.1.125.193:50723)
0810 07:20:30.143829 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:30.143864 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:30.143971 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:30.144229 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:30.144232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:30.144239 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:20:30.144383 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:30.144389 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.145132 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.145136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:30.145759 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:30.145767 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.146226 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.146229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:30.146308 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.146357 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:20:30.146640 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:20:30.146648 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:20:30.146713 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:20:30.146848 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":52,"thread_start_us":132,"threads_started":1}
I20260810 07:20:30.146986 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.143811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:50723 (local address 127.1.125.193:37879)
0810 07:20:30.143934 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:30.143938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:30.143950 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:20:30.144085 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:30.144089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:30.144129 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:30.144212 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:30.144506 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.145023 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.145884 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.146118 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.146225 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.146487 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:30.146560 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:30.146638 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:30.146722 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 07:20:30.146828 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":52}
I20260810 07:20:30.153913 13528 raft_consensus.cc:1261] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Refusing update from remote peer e6901a7f9e3e4f4d902b29bc7a63dc99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:30.154014 13332 raft_consensus.cc:1261] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Refusing update from remote peer e6901a7f9e3e4f4d902b29bc7a63dc99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:30.154312 13655 consensus_queue.cc:1035] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:30.154800 13655 consensus_queue.cc:1035] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:30.196530 12905 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:30.196705 12905 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:30.198802 12905 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d: Sending DeleteTablet for 3 replicas of tablet cddb0d5e2b264417a5dc6304c171fd33
I20260810 07:20:30.199447 13501 tablet_service.cc:1514] Processing DeleteTablet for tablet cddb0d5e2b264417a5dc6304c171fd33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:30 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:44398
I20260810 07:20:30.199470 13109 tablet_service.cc:1514] Processing DeleteTablet for tablet cddb0d5e2b264417a5dc6304c171fd33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:30 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:50934
I20260810 07:20:30.199664 13675 tablet_replica.cc:330] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99: stopping tablet replica
I20260810 07:20:30.199666 13677 tablet_replica.cc:330] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2: stopping tablet replica
I20260810 07:20:30.199779 13675 raft_consensus.cc:2229] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:30.199779 13677 raft_consensus.cc:2229] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:30.199909 13677 raft_consensus.cc:2258] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:30.199965 13675 raft_consensus.cc:2258] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:30.200318 13304 tablet_service.cc:1514] Processing DeleteTablet for tablet cddb0d5e2b264417a5dc6304c171fd33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:30 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:37182
I20260810 07:20:30.200551 13678 tablet_replica.cc:330] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1: stopping tablet replica
I20260810 07:20:30.200671 13678 raft_consensus.cc:2229] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:30.200744 13675 ts_tablet_manager.cc:1905] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:30.200769 13678 raft_consensus.cc:2258] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:30.201462 13678 ts_tablet_manager.cc:1905] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:30.204154 13678 ts_tablet_manager.cc:1918] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:20:30.204237 13678 log.cc:1196] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/cddb0d5e2b264417a5dc6304c171fd33
I20260810 07:20:30.204505 13678 ts_tablet_manager.cc:1939] T cddb0d5e2b264417a5dc6304c171fd33 P 31e1c407fdca451f95315081115ce9c1: Deleting consensus metadata
I20260810 07:20:30.204864 13675 ts_tablet_manager.cc:1918] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:20:30.205032 13675 log.cc:1196] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/wal/wals/cddb0d5e2b264417a5dc6304c171fd33
I20260810 07:20:30.205389 13675 ts_tablet_manager.cc:1939] T cddb0d5e2b264417a5dc6304c171fd33 P e6901a7f9e3e4f4d902b29bc7a63dc99: Deleting consensus metadata
I20260810 07:20:30.205665 12886 catalog_manager.cc:4841] TS 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403): tablet cddb0d5e2b264417a5dc6304c171fd33 (table test-table [id=4eaa18bbf18745348de0507fa6949c0a]) successfully deleted
I20260810 07:20:30.206223 12887 catalog_manager.cc:4841] TS e6901a7f9e3e4f4d902b29bc7a63dc99 (127.1.125.193:37879): tablet cddb0d5e2b264417a5dc6304c171fd33 (table test-table [id=4eaa18bbf18745348de0507fa6949c0a]) successfully deleted
I20260810 07:20:30.206442 13677 ts_tablet_manager.cc:1905] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:30.209115 13677 ts_tablet_manager.cc:1918] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:20:30.209209 13677 log.cc:1196] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/cddb0d5e2b264417a5dc6304c171fd33
I20260810 07:20:30.209448 13677 ts_tablet_manager.cc:1939] T cddb0d5e2b264417a5dc6304c171fd33 P 117d2c7dbd9543e68149dc64fc6ba6a2: Deleting consensus metadata
I20260810 07:20:30.210155 12886 catalog_manager.cc:4841] TS 117d2c7dbd9543e68149dc64fc6ba6a2 (127.1.125.195:39575): tablet cddb0d5e2b264417a5dc6304c171fd33 (table test-table [id=4eaa18bbf18745348de0507fa6949c0a]) successfully deleted
Aug 10 07:20:30 dist-test-slave-44d2 krb5kdc[12841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:20:30 dist-test-slave-44d2 krb5kdc[12841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:30 dist-test-slave-44d2 krb5kdc[12841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:20:30.239420 13690 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:30.232627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58562 (local address 127.1.125.254:34167)
0810 07:20:30.232780 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:30.232785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:30.232959 (+ 174us) server_negotiation.cc:407] Connection header received
0810 07:20:30.233154 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:30.233157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:30.233211 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:30.233301 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:30.234053 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.234601 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.235490 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.235734 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.235870 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.239287 (+ 3417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58562: BlockingRecv error: recv got EOF from 127.0.0.1:58562 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 07:20:30 dist-test-slave-44d2 krb5kdc[12841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM
I20260810 07:20:30.250653 13690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.241304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58564 (local address 127.1.125.254:34167)
0810 07:20:30.241447 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:30.241451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:30.241706 (+ 255us) server_negotiation.cc:407] Connection header received
0810 07:20:30.241801 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:30.241803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:30.241844 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:30.241910 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:30.242704 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.243196 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.244368 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.244655 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.244735 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.247278 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:30.247282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:30.247284 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:30.247307 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:30.248937 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:30.249368 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:30.249371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:30.249374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:30.249460 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:30.249751 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:30.249755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:30.249756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:30.249939 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:30.250083 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:30.250331 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 07:20:30.250466 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:20:30.253577 12905 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:20:30.254062 12905 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:30.258942 13109 tablet_service.cc:1467] Processing CreateTablet for tablet a824a4fff02f45e5a2c0b934e1cdf469 (DEFAULT_TABLE table=test-table [id=5b922c501b2c42aa8daf7a70097ff8c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:30.259028 13304 tablet_service.cc:1467] Processing CreateTablet for tablet a824a4fff02f45e5a2c0b934e1cdf469 (DEFAULT_TABLE table=test-table [id=5b922c501b2c42aa8daf7a70097ff8c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:30.259246 13109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a824a4fff02f45e5a2c0b934e1cdf469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:30.259351 13304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a824a4fff02f45e5a2c0b934e1cdf469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:30.262118 13643 tablet_bootstrap.cc:492] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1: Bootstrap starting.
I20260810 07:20:30.262462 13642 tablet_bootstrap.cc:492] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99: Bootstrap starting.
I20260810 07:20:30.262832 13501 tablet_service.cc:1467] Processing CreateTablet for tablet a824a4fff02f45e5a2c0b934e1cdf469 (DEFAULT_TABLE table=test-table [id=5b922c501b2c42aa8daf7a70097ff8c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:30.263144 13501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a824a4fff02f45e5a2c0b934e1cdf469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:30.263139 13643 tablet_bootstrap.cc:654] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:30.263382 13642 tablet_bootstrap.cc:654] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:30.265048 13642 tablet_bootstrap.cc:492] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99: No bootstrap required, opened a new log
I20260810 07:20:30.265137 13642 ts_tablet_manager.cc:1397] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:20:30.265630 13641 tablet_bootstrap.cc:492] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2: Bootstrap starting.
I20260810 07:20:30.265601 13642 raft_consensus.cc:348] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.265704 13642 raft_consensus.cc:374] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:30.265733 13642 raft_consensus.cc:729] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6901a7f9e3e4f4d902b29bc7a63dc99, State: Initialized, Role: FOLLOWER
I20260810 07:20:30.265851 13642 consensus_queue.cc:260] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.266119 13642 ts_tablet_manager.cc:1428] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:30.266535 13641 tablet_bootstrap.cc:654] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:30.266618 13643 tablet_bootstrap.cc:492] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1: No bootstrap required, opened a new log
I20260810 07:20:30.266701 13643 ts_tablet_manager.cc:1397] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260810 07:20:30.267194 13643 raft_consensus.cc:348] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.267297 13643 raft_consensus.cc:374] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:30.267336 13643 raft_consensus.cc:729] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e1c407fdca451f95315081115ce9c1, State: Initialized, Role: FOLLOWER
I20260810 07:20:30.267459 13643 consensus_queue.cc:260] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.267768 13643 ts_tablet_manager.cc:1428] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:30.269577 13641 tablet_bootstrap.cc:492] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2: No bootstrap required, opened a new log
I20260810 07:20:30.269690 13641 ts_tablet_manager.cc:1397] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260810 07:20:30.270148 13641 raft_consensus.cc:348] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.270274 13641 raft_consensus.cc:374] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:30.270321 13641 raft_consensus.cc:729] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d2c7dbd9543e68149dc64fc6ba6a2, State: Initialized, Role: FOLLOWER
I20260810 07:20:30.270458 13641 consensus_queue.cc:260] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.271011 13641 ts_tablet_manager.cc:1428] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:30.302809 13649 raft_consensus.cc:482] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:30.302956 13649 raft_consensus.cc:504] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.304126 13649 leader_election.cc:288] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6901a7f9e3e4f4d902b29bc7a63dc99 (127.1.125.193:37879), 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403)
I20260810 07:20:30.308856 13740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.304510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:44659 (local address 127.1.125.194:43403)
0810 07:20:30.304787 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:30.304792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:30.304923 (+ 131us) server_negotiation.cc:407] Connection header received
0810 07:20:30.305123 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:30.305125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:30.305170 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:30.305259 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:30.305579 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.306157 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.307459 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.308134 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.308296 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.308352 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:30.308630 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 07:20:30.308710 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":96,"threads_started":1}
I20260810 07:20:30.308861 13736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.304363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:43403 (local address 127.1.125.195:44659)
0810 07:20:30.304827 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:30.304849 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:30.305005 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:30.305281 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:30.305284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:30.305302 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:30.305458 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:30.305464 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.306293 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.306297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:30.307287 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:30.307297 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.308300 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.308304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:30.308455 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.308507 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 07:20:30.308604 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 07:20:30.308685 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":111,"spinlock_wait_cycles":25088,"thread_start_us":132,"threads_started":1}
I20260810 07:20:30.308912 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.304530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:36259 (local address 127.1.125.193:37879)
0810 07:20:30.304677 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:30.304682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:30.304929 (+ 247us) server_negotiation.cc:407] Connection header received
0810 07:20:30.305145 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:30.305148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:30.305193 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:30.305283 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:30.305937 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.306433 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.307554 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.308186 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.308299 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.308352 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:30.308724 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 07:20:30.308808 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:20:30.308930 13738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.304363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:37879 (local address 127.1.125.195:36259)
0810 07:20:30.304811 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:30.304824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:30.305003 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:30.305293 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:30.305297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:30.305314 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:30.305800 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:30.305808 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.306548 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.306554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:30.307374 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:30.307404 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.308415 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.308419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:30.308593 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.308620 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 07:20:30.308701 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 07:20:30.308770 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":32,"spinlock_wait_cycles":32384,"thread_start_us":111,"threads_started":1}
I20260810 07:20:30.309625 13135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a824a4fff02f45e5a2c0b934e1cdf469" candidate_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" is_pre_election: true
I20260810 07:20:30.309732 13331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a824a4fff02f45e5a2c0b934e1cdf469" candidate_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e1c407fdca451f95315081115ce9c1" is_pre_election: true
I20260810 07:20:30.309895 13331 raft_consensus.cc:2454] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117d2c7dbd9543e68149dc64fc6ba6a2 in term 0.
I20260810 07:20:30.309906 13135 raft_consensus.cc:2454] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117d2c7dbd9543e68149dc64fc6ba6a2 in term 0.
I20260810 07:20:30.310318 13432 leader_election.cc:302] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 117d2c7dbd9543e68149dc64fc6ba6a2, 31e1c407fdca451f95315081115ce9c1, e6901a7f9e3e4f4d902b29bc7a63dc99; no voters:
I20260810 07:20:30.310634 13649 raft_consensus.cc:2781] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:30.310714 13649 raft_consensus.cc:482] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:30.310770 13649 raft_consensus.cc:3037] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:30.311825 13649 raft_consensus.cc:504] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.312196 13649 leader_election.cc:288] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [CANDIDATE]: Term 1 election: Requested vote from peers e6901a7f9e3e4f4d902b29bc7a63dc99 (127.1.125.193:37879), 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403)
I20260810 07:20:30.312613 13135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a824a4fff02f45e5a2c0b934e1cdf469" candidate_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99"
I20260810 07:20:30.312621 13331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a824a4fff02f45e5a2c0b934e1cdf469" candidate_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e1c407fdca451f95315081115ce9c1"
I20260810 07:20:30.312744 13331 raft_consensus.cc:3037] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:30.312744 13135 raft_consensus.cc:3037] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:30.314056 13135 raft_consensus.cc:2454] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d2c7dbd9543e68149dc64fc6ba6a2 in term 1.
I20260810 07:20:30.314391 13435 leader_election.cc:302] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d2c7dbd9543e68149dc64fc6ba6a2, e6901a7f9e3e4f4d902b29bc7a63dc99; no voters:
I20260810 07:20:30.314415 13331 raft_consensus.cc:2454] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d2c7dbd9543e68149dc64fc6ba6a2 in term 1.
I20260810 07:20:30.314599 13649 raft_consensus.cc:2781] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:30.314860 13649 raft_consensus.cc:686] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 LEADER]: Becoming Leader. State: Replica: 117d2c7dbd9543e68149dc64fc6ba6a2, State: Running, Role: LEADER
I20260810 07:20:30.315140 13649 consensus_queue.cc:237] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.317440 12903 catalog_manager.cc:5495] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 117d2c7dbd9543e68149dc64fc6ba6a2 (127.1.125.195). New cstate: current_term: 1 leader_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } health_report { overall_health: HEALTHY } } }
I20260810 07:20:30.328131 13746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.324354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47572 (local address 127.1.125.195:39575)
0810 07:20:30.324675 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:30.324683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:30.324693 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:20:30.324756 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:30.324758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:30.324806 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:30.324931 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:30.325198 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.325727 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.326611 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.326861 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.326971 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.327263 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:30.327721 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:30.327855 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:30.327913 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 07:20:30.327967 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":94,"threads_started":1}
I20260810 07:20:30.330822 13331 raft_consensus.cc:1261] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Refusing update from remote peer 117d2c7dbd9543e68149dc64fc6ba6a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:30.330960 13135 raft_consensus.cc:1261] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Refusing update from remote peer 117d2c7dbd9543e68149dc64fc6ba6a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:30.331315 13741 consensus_queue.cc:1035] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:30.331511 13649 consensus_queue.cc:1035] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:30.342758 12905 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:30.342886 12905 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:30.344923 12905 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d: Sending DeleteTablet for 3 replicas of tablet a824a4fff02f45e5a2c0b934e1cdf469
I20260810 07:20:30.345466 13109 tablet_service.cc:1514] Processing DeleteTablet for tablet a824a4fff02f45e5a2c0b934e1cdf469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:30 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:50934
I20260810 07:20:30.345652 13304 tablet_service.cc:1514] Processing DeleteTablet for tablet a824a4fff02f45e5a2c0b934e1cdf469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:30 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:37182
I20260810 07:20:30.345679 13675 tablet_replica.cc:330] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99: stopping tablet replica
I20260810 07:20:30.345822 13678 tablet_replica.cc:330] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1: stopping tablet replica
I20260810 07:20:30.345821 13675 raft_consensus.cc:2229] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:30.345928 13678 raft_consensus.cc:2229] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:30.345958 13675 raft_consensus.cc:2258] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:30.346041 13678 raft_consensus.cc:2258] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:30.346341 12905 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:20:30.346730 13501 tablet_service.cc:1514] Processing DeleteTablet for tablet a824a4fff02f45e5a2c0b934e1cdf469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:30 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:44398
W20260810 07:20:30.346797 12905 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:30.346901 13677 tablet_replica.cc:330] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2: stopping tablet replica
I20260810 07:20:30.347014 13677 raft_consensus.cc:2229] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:30.347290 13677 raft_consensus.cc:2258] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:30.347939 13677 ts_tablet_manager.cc:1905] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:30.347939 13675 ts_tablet_manager.cc:1905] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:30.350616 13675 ts_tablet_manager.cc:1918] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:30.350616 13677 ts_tablet_manager.cc:1918] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:30.350695 13677 log.cc:1196] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/a824a4fff02f45e5a2c0b934e1cdf469
I20260810 07:20:30.350695 13675 log.cc:1196] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-0/wal/wals/a824a4fff02f45e5a2c0b934e1cdf469
I20260810 07:20:30.350963 13677 ts_tablet_manager.cc:1939] T a824a4fff02f45e5a2c0b934e1cdf469 P 117d2c7dbd9543e68149dc64fc6ba6a2: Deleting consensus metadata
I20260810 07:20:30.350963 13675 ts_tablet_manager.cc:1939] T a824a4fff02f45e5a2c0b934e1cdf469 P e6901a7f9e3e4f4d902b29bc7a63dc99: Deleting consensus metadata
I20260810 07:20:30.351719 12886 catalog_manager.cc:4841] TS 117d2c7dbd9543e68149dc64fc6ba6a2 (127.1.125.195:39575): tablet a824a4fff02f45e5a2c0b934e1cdf469 (table test-table [id=5b922c501b2c42aa8daf7a70097ff8c3]) successfully deleted
I20260810 07:20:30.352043 12887 catalog_manager.cc:4841] TS e6901a7f9e3e4f4d902b29bc7a63dc99 (127.1.125.193:37879): tablet a824a4fff02f45e5a2c0b934e1cdf469 (table test-table [id=5b922c501b2c42aa8daf7a70097ff8c3]) successfully deleted
I20260810 07:20:30.352713 13678 ts_tablet_manager.cc:1905] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:30.355037 13678 ts_tablet_manager.cc:1918] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:30.355123 13678 log.cc:1196] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/a824a4fff02f45e5a2c0b934e1cdf469
I20260810 07:20:30.355404 13678 ts_tablet_manager.cc:1939] T a824a4fff02f45e5a2c0b934e1cdf469 P 31e1c407fdca451f95315081115ce9c1: Deleting consensus metadata
I20260810 07:20:30.355557 13304 tablet_service.cc:1467] Processing CreateTablet for tablet 27c31580ca234a31a1c5cf6deadc37b2 (DEFAULT_TABLE table=test-table [id=1d2adc2230254c4783e2c3ec9bd9ce1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:30.355659 13501 tablet_service.cc:1467] Processing CreateTablet for tablet 27c31580ca234a31a1c5cf6deadc37b2 (DEFAULT_TABLE table=test-table [id=1d2adc2230254c4783e2c3ec9bd9ce1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:30.355804 13304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27c31580ca234a31a1c5cf6deadc37b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:30.355907 13501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27c31580ca234a31a1c5cf6deadc37b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:30.356564 12886 catalog_manager.cc:4841] TS 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403): tablet a824a4fff02f45e5a2c0b934e1cdf469 (table test-table [id=5b922c501b2c42aa8daf7a70097ff8c3]) successfully deleted
I20260810 07:20:30.357936 13643 tablet_bootstrap.cc:492] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1: Bootstrap starting.
I20260810 07:20:30.358233 13641 tablet_bootstrap.cc:492] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2: Bootstrap starting.
I20260810 07:20:30.358922 13643 tablet_bootstrap.cc:654] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:30.359196 13641 tablet_bootstrap.cc:654] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:30.360059 13109 tablet_service.cc:1467] Processing CreateTablet for tablet 27c31580ca234a31a1c5cf6deadc37b2 (DEFAULT_TABLE table=test-table [id=1d2adc2230254c4783e2c3ec9bd9ce1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:30.360239 13643 tablet_bootstrap.cc:492] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1: No bootstrap required, opened a new log
I20260810 07:20:30.360316 13643 ts_tablet_manager.cc:1397] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.001s
I20260810 07:20:30.360330 13109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27c31580ca234a31a1c5cf6deadc37b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:30.360812 13643 raft_consensus.cc:348] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.360929 13643 raft_consensus.cc:374] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:30.360958 13643 raft_consensus.cc:729] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e1c407fdca451f95315081115ce9c1, State: Initialized, Role: FOLLOWER
I20260810 07:20:30.361074 13643 consensus_queue.cc:260] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.361447 13643 ts_tablet_manager.cc:1428] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:30.362840 13642 tablet_bootstrap.cc:492] T 27c31580ca234a31a1c5cf6deadc37b2 P e6901a7f9e3e4f4d902b29bc7a63dc99: Bootstrap starting.
I20260810 07:20:30.363543 13641 tablet_bootstrap.cc:492] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2: No bootstrap required, opened a new log
I20260810 07:20:30.363641 13641 ts_tablet_manager.cc:1397] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260810 07:20:30.364035 13642 tablet_bootstrap.cc:654] T 27c31580ca234a31a1c5cf6deadc37b2 P e6901a7f9e3e4f4d902b29bc7a63dc99: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:30.364107 13641 raft_consensus.cc:348] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.364238 13641 raft_consensus.cc:374] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:30.364280 13641 raft_consensus.cc:729] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117d2c7dbd9543e68149dc64fc6ba6a2, State: Initialized, Role: FOLLOWER
I20260810 07:20:30.364400 13641 consensus_queue.cc:260] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.364776 13641 ts_tablet_manager.cc:1428] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:30.366031 13642 tablet_bootstrap.cc:492] T 27c31580ca234a31a1c5cf6deadc37b2 P e6901a7f9e3e4f4d902b29bc7a63dc99: No bootstrap required, opened a new log
I20260810 07:20:30.366114 13642 ts_tablet_manager.cc:1397] T 27c31580ca234a31a1c5cf6deadc37b2 P e6901a7f9e3e4f4d902b29bc7a63dc99: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:20:30.366494 13642 raft_consensus.cc:348] T 27c31580ca234a31a1c5cf6deadc37b2 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.366592 13642 raft_consensus.cc:374] T 27c31580ca234a31a1c5cf6deadc37b2 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:30.366622 13642 raft_consensus.cc:729] T 27c31580ca234a31a1c5cf6deadc37b2 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6901a7f9e3e4f4d902b29bc7a63dc99, State: Initialized, Role: FOLLOWER
I20260810 07:20:30.366739 13642 consensus_queue.cc:260] T 27c31580ca234a31a1c5cf6deadc37b2 P e6901a7f9e3e4f4d902b29bc7a63dc99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.366979 13642 ts_tablet_manager.cc:1428] T 27c31580ca234a31a1c5cf6deadc37b2 P e6901a7f9e3e4f4d902b29bc7a63dc99: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:20:30.398867 13750 raft_consensus.cc:482] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:30.399024 13750 raft_consensus.cc:504] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.399497 13750 leader_election.cc:288] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6901a7f9e3e4f4d902b29bc7a63dc99 (127.1.125.193:37879), 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403)
I20260810 07:20:30.399919 13331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27c31580ca234a31a1c5cf6deadc37b2" candidate_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e1c407fdca451f95315081115ce9c1" is_pre_election: true
I20260810 07:20:30.399926 13135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27c31580ca234a31a1c5cf6deadc37b2" candidate_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" is_pre_election: true
I20260810 07:20:30.400075 13331 raft_consensus.cc:2454] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117d2c7dbd9543e68149dc64fc6ba6a2 in term 0.
I20260810 07:20:30.400108 13135 raft_consensus.cc:2454] T 27c31580ca234a31a1c5cf6deadc37b2 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117d2c7dbd9543e68149dc64fc6ba6a2 in term 0.
I20260810 07:20:30.400403 13432 leader_election.cc:302] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d2c7dbd9543e68149dc64fc6ba6a2, 31e1c407fdca451f95315081115ce9c1; no voters:
I20260810 07:20:30.400620 13750 raft_consensus.cc:2781] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:30.400713 13750 raft_consensus.cc:482] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:30.400772 13750 raft_consensus.cc:3037] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:30.402304 13750 raft_consensus.cc:504] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.402722 13750 leader_election.cc:288] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [CANDIDATE]: Term 1 election: Requested vote from peers e6901a7f9e3e4f4d902b29bc7a63dc99 (127.1.125.193:37879), 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403)
I20260810 07:20:30.403025 13135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27c31580ca234a31a1c5cf6deadc37b2" candidate_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99"
I20260810 07:20:30.403160 13135 raft_consensus.cc:3037] T 27c31580ca234a31a1c5cf6deadc37b2 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:30.404408 13135 raft_consensus.cc:2454] T 27c31580ca234a31a1c5cf6deadc37b2 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d2c7dbd9543e68149dc64fc6ba6a2 in term 1.
I20260810 07:20:30.404709 13331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27c31580ca234a31a1c5cf6deadc37b2" candidate_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e1c407fdca451f95315081115ce9c1"
I20260810 07:20:30.404829 13331 raft_consensus.cc:3037] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:30.404969 13435 leader_election.cc:302] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117d2c7dbd9543e68149dc64fc6ba6a2, e6901a7f9e3e4f4d902b29bc7a63dc99; no voters:
I20260810 07:20:30.405134 13750 raft_consensus.cc:2781] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:30.405293 13750 raft_consensus.cc:686] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 LEADER]: Becoming Leader. State: Replica: 117d2c7dbd9543e68149dc64fc6ba6a2, State: Running, Role: LEADER
I20260810 07:20:30.405448 13750 consensus_queue.cc:237] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } }
I20260810 07:20:30.405952 13331 raft_consensus.cc:2454] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117d2c7dbd9543e68149dc64fc6ba6a2 in term 1.
I20260810 07:20:30.407500 12905 catalog_manager.cc:5495] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 117d2c7dbd9543e68149dc64fc6ba6a2 (127.1.125.195). New cstate: current_term: 1 leader_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117d2c7dbd9543e68149dc64fc6ba6a2" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39575 } health_report { overall_health: HEALTHY } } }
I20260810 07:20:30.436115 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.432439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:33341 (local address 127.1.125.193:37879)
0810 07:20:30.432607 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:30.432612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:30.432802 (+ 190us) server_negotiation.cc:407] Connection header received
0810 07:20:30.432898 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:30.432900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:30.432954 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:30.433036 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:30.433256 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.434057 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.434886 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.435213 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.435324 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.435659 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:30.435743 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:30.435824 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:30.435908 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 07:20:30.435961 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 07:20:30.436240 13760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.432363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:37879 (local address 127.1.125.195:33341)
0810 07:20:30.432718 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:30.432732 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:30.432824 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:30.433053 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:30.433055 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:30.433061 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:20:30.433160 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:30.433164 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.434160 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.434162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:30.434766 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:30.434774 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.435354 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.435359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:30.435446 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.435503 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:20:30.435837 (+ 334us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:20:30.435843 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:20:30.435895 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 07:20:30.435962 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":142,"threads_started":1}
I20260810 07:20:30.442698 13135 raft_consensus.cc:1261] T 27c31580ca234a31a1c5cf6deadc37b2 P e6901a7f9e3e4f4d902b29bc7a63dc99 [term 1 FOLLOWER]: Refusing update from remote peer 117d2c7dbd9543e68149dc64fc6ba6a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:30.443125 13649 consensus_queue.cc:1035] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6901a7f9e3e4f4d902b29bc7a63dc99" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:30.444136 13331 raft_consensus.cc:1261] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Refusing update from remote peer 117d2c7dbd9543e68149dc64fc6ba6a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:30.445027 13649 consensus_queue.cc:1035] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31e1c407fdca451f95315081115ce9c1" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:30.462375 13711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.458530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:39575 (local address 127.1.125.193:34705)
0810 07:20:30.458669 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:30.458686 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:30.458770 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:30.459330 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:30.459333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:30.459342 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:20:30.459452 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:30.459459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.460341 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.460346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:30.461051 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:30.461062 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.461649 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.461654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:30.461752 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.461788 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:20:30.462090 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:20:30.462098 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:20:30.462165 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 07:20:30.462264 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 07:20:30.462369 13746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:30.459006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:34705 (local address 127.1.125.195:39575)
0810 07:20:30.459113 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:30.459118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:30.459132 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:20:30.459174 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:30.459177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:30.459229 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:30.459318 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:30.459571 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.460228 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.461194 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:30.461532 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:30.461654 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:30.461914 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:30.461986 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:30.462072 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:30.462175 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 07:20:30.462229 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:20:30.487296 12903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:30.487498 12903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:30.493428 12903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18e430a9a9734a9ba37b4cdd4ca8b83d: Sending DeleteTablet for 3 replicas of tablet 27c31580ca234a31a1c5cf6deadc37b2
I20260810 07:20:30.494417 13304 tablet_service.cc:1514] Processing DeleteTablet for tablet 27c31580ca234a31a1c5cf6deadc37b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:30 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:37182
I20260810 07:20:30.494551 13501 tablet_service.cc:1514] Processing DeleteTablet for tablet 27c31580ca234a31a1c5cf6deadc37b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:30 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:44398
I20260810 07:20:30.494668 13678 tablet_replica.cc:330] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1: stopping tablet replica
I20260810 07:20:30.494781 13677 tablet_replica.cc:330] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2: stopping tablet replica
I20260810 07:20:30.494801 13678 raft_consensus.cc:2229] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:30.494930 13678 raft_consensus.cc:2258] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:30.494941 13677 raft_consensus.cc:2229] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:30.495185 13677 raft_consensus.cc:2258] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:30.495661 13109 tablet_service.cc:1514] Processing DeleteTablet for tablet 27c31580ca234a31a1c5cf6deadc37b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:30 UTC) from {username='oryx', principal='oryx/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:50934
I20260810 07:20:30.495800 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 13016
I20260810 07:20:30.517843 13678 ts_tablet_manager.cc:1905] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:30.519279 13677 ts_tablet_manager.cc:1905] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:30.520584 13678 ts_tablet_manager.cc:1918] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:20:30.520666 13678 log.cc:1196] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/27c31580ca234a31a1c5cf6deadc37b2
I20260810 07:20:30.520924 13678 ts_tablet_manager.cc:1939] T 27c31580ca234a31a1c5cf6deadc37b2 P 31e1c407fdca451f95315081115ce9c1: Deleting consensus metadata
I20260810 07:20:30.521734 12886 catalog_manager.cc:4841] TS 31e1c407fdca451f95315081115ce9c1 (127.1.125.194:43403): tablet 27c31580ca234a31a1c5cf6deadc37b2 (table test-table [id=1d2adc2230254c4783e2c3ec9bd9ce1d]) successfully deleted
I20260810 07:20:30.522320 13677 ts_tablet_manager.cc:1918] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:20:30.522413 13677 log.cc:1196] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/27c31580ca234a31a1c5cf6deadc37b2
I20260810 07:20:30.522727 13677 ts_tablet_manager.cc:1939] T 27c31580ca234a31a1c5cf6deadc37b2 P 117d2c7dbd9543e68149dc64fc6ba6a2: Deleting consensus metadata
W20260810 07:20:30.523140 12887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:37879 (error 108)
I20260810 07:20:30.523679 12886 catalog_manager.cc:4841] TS 117d2c7dbd9543e68149dc64fc6ba6a2 (127.1.125.195:39575): tablet 27c31580ca234a31a1c5cf6deadc37b2 (table test-table [id=1d2adc2230254c4783e2c3ec9bd9ce1d]) successfully deleted
I20260810 07:20:30.524509 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 13213
W20260810 07:20:30.524569 13764 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:30.523972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:37879 (local address 127.0.0.1:45012)
0810 07:20:30.524357 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:30.524427 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:37879: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":88,"threads_started":1}
W20260810 07:20:30.529424 12887 catalog_manager.cc:4584] TS e6901a7f9e3e4f4d902b29bc7a63dc99 (127.1.125.193:37879): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 27c31580ca234a31a1c5cf6deadc37b2: Network error: Client connection negotiation failed: client connection to 127.1.125.193:37879: connect: Connection reset by peer (error 104)
I20260810 07:20:30.539744 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 13406
I20260810 07:20:30.555668 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 12860
2026-08-10T07:20:30Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3781 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:30 dist-test-slave-44d2 krb5kdc[13768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:30 dist-test-slave-44d2 krb5kdc[13768](info): set up 2 sockets
Aug 10 07:20:30 dist-test-slave-44d2 krb5kdc[13768](info): commencing operation
krb5kdc: starting...
W20260810 07:20:31.855288 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:20:31 dist-test-slave-44d2 krb5kdc[13768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:20:31.984216 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:42215
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:38617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:42215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:32.098007 13788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:32.098275 13788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:32.098315 13788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:32.102698 13788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:32.102766 13788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:32.102787 13788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:32.102942 13788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:32.102983 13788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:32.108456 13788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:38617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:42215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:42215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:32.109591 13788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:32.110649 13788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:32.119387 13802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:32.119526 13788 server_base.cc:1034] running on GCE node
W20260810 07:20:32.119376 13801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:32.119768 13807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:32.120337 13788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:32.121587 13788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:32.122764 13788 hybrid_clock.cc:648] HybridClock initialized: now 1786346432122734 us; error 41 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T07:20:32Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1567 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:32 dist-test-slave-44d2 krb5kdc[13813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:32 dist-test-slave-44d2 krb5kdc[13813](info): set up 2 sockets
Aug 10 07:20:32 dist-test-slave-44d2 krb5kdc[13813](info): commencing operation
krb5kdc: starting...
W20260810 07:20:33.380862 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:20:33 dist-test-slave-44d2 krb5kdc[13813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:20:33.519073 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:40525
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:40525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:33.634873 13828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:33.635134 13828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:33.635211 13828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:33.639266 13828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:33.639333 13828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:33.639353 13828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:33.639371 13828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:33.639386 13828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:33.644701 13828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:40525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:40525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:33.645890 13828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:33.646960 13828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:33.655830 13841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:33.655885 13838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:33.655829 13835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:33.656133 13828 server_base.cc:1034] running on GCE node
I20260810 07:20:33.656859 13828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:33.657982 13828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:33.659142 13828 hybrid_clock.cc:648] HybridClock initialized: now 1786346433659118 us; error 41 us; skew 500 ppm
Aug 10 07:20:33 dist-test-slave-44d2 krb5kdc[13813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346433, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:33.666184 13828 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:20:33.667075 13828 webserver.cc:460] Webserver started at http://127.1.125.254:36781/ using document root <none> and password file <none>
I20260810 07:20:33.667618 13828 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:33.667678 13828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:33.667845 13828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:33.669905 13828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "1b52442a4da444599fc49b69ccfcc750"
format_stamp: "Formatted at 2026-08-10 07:20:33 on dist-test-slave-44d2"
server_key: "ae7867f7ceefc7b81f7fb275741be824"
server_key_iv: "6f6e4cd69edcd61a1ef06e7bc7d92006"
server_key_version: "encryptionkey@0"
I20260810 07:20:33.670420 13828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "1b52442a4da444599fc49b69ccfcc750"
format_stamp: "Formatted at 2026-08-10 07:20:33 on dist-test-slave-44d2"
server_key: "ae7867f7ceefc7b81f7fb275741be824"
server_key_iv: "6f6e4cd69edcd61a1ef06e7bc7d92006"
server_key_version: "encryptionkey@0"
I20260810 07:20:33.674129 13828 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260810 07:20:33.676602 13849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:33.677887 13828 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:20:33.678038 13828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "1b52442a4da444599fc49b69ccfcc750"
format_stamp: "Formatted at 2026-08-10 07:20:33 on dist-test-slave-44d2"
server_key: "ae7867f7ceefc7b81f7fb275741be824"
server_key_iv: "6f6e4cd69edcd61a1ef06e7bc7d92006"
server_key_version: "encryptionkey@0"
I20260810 07:20:33.678169 13828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:33.691205 13828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:33.694322 13828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:33.694583 13828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:33.702118 13828 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:40525
I20260810 07:20:33.702148 13916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:40525 every 8 connection(s)
I20260810 07:20:33.703099 13828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:33.705719 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 13828
I20260810 07:20:33.705868 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:33.706159 13918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:33.706244 1527 external_mini_cluster.cc:1428] Setting key 84524ddde4c5ed923555985f5e31c20e
I20260810 07:20:33.713029 13918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750: Bootstrap starting.
Aug 10 07:20:33 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:33.715597 13918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:33.716552 13918 log.cc:824] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:33.718902 13918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750: No bootstrap required, opened a new log
I20260810 07:20:33.720986 13921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:33.707732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41384 (local address 127.1.125.254:40525)
0810 07:20:33.708180 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:33.708195 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:33.708230 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:20:33.709125 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:33.709146 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:33.709471 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:33.709850 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:33.710698 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:33.711686 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:33.712627 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:33.713146 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:33.713334 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:33.716135 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:33.716141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:33.716154 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:33.716181 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:33.718028 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:33.718543 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:33.718548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:33.718553 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:33.718660 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:33.719009 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:33.719015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:33.719017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:33.719411 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:33.719563 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:33.719868 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 07:20:33.720195 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":144,"threads_started":1}
I20260810 07:20:33.723135 13918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b52442a4da444599fc49b69ccfcc750" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40525 } }
I20260810 07:20:33.723414 13918 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:33.723482 13918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b52442a4da444599fc49b69ccfcc750, State: Initialized, Role: FOLLOWER
I20260810 07:20:33.723918 13918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b52442a4da444599fc49b69ccfcc750" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40525 } }
I20260810 07:20:33.724068 13918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:33.724138 13918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:33.724236 13918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:33.725226 13918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b52442a4da444599fc49b69ccfcc750" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40525 } }
I20260810 07:20:33.725587 13918 leader_election.cc:302] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b52442a4da444599fc49b69ccfcc750; no voters:
I20260810 07:20:33.725919 13918 leader_election.cc:288] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:33.726083 13924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:33.726336 13924 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 1 LEADER]: Becoming Leader. State: Replica: 1b52442a4da444599fc49b69ccfcc750, State: Running, Role: LEADER
I20260810 07:20:33.726658 13924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b52442a4da444599fc49b69ccfcc750" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40525 } }
I20260810 07:20:33.727258 13918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:33.728320 13926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b52442a4da444599fc49b69ccfcc750. Latest consensus state: current_term: 1 leader_uuid: "1b52442a4da444599fc49b69ccfcc750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b52442a4da444599fc49b69ccfcc750" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40525 } } }
I20260810 07:20:33.728462 13926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:33.728438 13925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b52442a4da444599fc49b69ccfcc750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b52442a4da444599fc49b69ccfcc750" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40525 } } }
I20260810 07:20:33.728552 13925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [sys.catalog]: This master's current role is: LEADER
W20260810 07:20:33.732986 13943 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:20:33.733085 13943 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:20:33.733147 13934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:33.734090 13934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:33.740259 13934 catalog_manager.cc:1347] Generated new cluster ID: 44487248a315429197364e1970c82d71
I20260810 07:20:33.740343 13934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:33.749200 13934 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:33.750133 13934 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:33.757359 13934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750: Generated new TSK 0
I20260810 07:20:33.758023 13934 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:20:33.797901 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:40525
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:33.927841 13953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:33.928114 13953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:33.928201 13953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:33.932544 13953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:33.932648 13953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:33.932751 13953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:33.938341 13953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:33.939586 13953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:33.940702 13953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:33.950668 13966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:33.950704 13962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:33.950668 13968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:33.950759 13953 server_base.cc:1034] running on GCE node
I20260810 07:20:33.951355 13953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:33.952003 13953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:33.953195 13953 hybrid_clock.cc:648] HybridClock initialized: now 1786346433953174 us; error 31 us; skew 500 ppm
Aug 10 07:20:33 dist-test-slave-44d2 krb5kdc[13813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346433, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:33.956082 13953 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:20:33.957108 13953 webserver.cc:460] Webserver started at http://127.1.125.193:34895/ using document root <none> and password file <none>
I20260810 07:20:33.957649 13953 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:33.957710 13953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:33.957880 13953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:33.959899 13953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "5eb02b5dcc9e4fe78cdd2e0c682f4226"
format_stamp: "Formatted at 2026-08-10 07:20:33 on dist-test-slave-44d2"
server_key: "321a487bc0791d99122f5212921ee112"
server_key_iv: "13f5fa31a267a1fc384cec793083edc1"
server_key_version: "encryptionkey@0"
I20260810 07:20:33.960408 13953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "5eb02b5dcc9e4fe78cdd2e0c682f4226"
format_stamp: "Formatted at 2026-08-10 07:20:33 on dist-test-slave-44d2"
server_key: "321a487bc0791d99122f5212921ee112"
server_key_iv: "13f5fa31a267a1fc384cec793083edc1"
server_key_version: "encryptionkey@0"
I20260810 07:20:33.964190 13953 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:33.966888 13980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:33.968122 13953 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:33.968251 13953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "5eb02b5dcc9e4fe78cdd2e0c682f4226"
format_stamp: "Formatted at 2026-08-10 07:20:33 on dist-test-slave-44d2"
server_key: "321a487bc0791d99122f5212921ee112"
server_key_iv: "13f5fa31a267a1fc384cec793083edc1"
server_key_version: "encryptionkey@0"
I20260810 07:20:33.968358 13953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:33.984225 13953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:33.987504 13953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:33.987722 13953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:33.988466 13953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:33.989440 13953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:33.989514 13953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:33.989578 13953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:33.989630 13953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:33.998621 13953 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:40617
I20260810 07:20:33.998704 14140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:40617 every 8 connection(s)
I20260810 07:20:33.999550 13953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:34.007210 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 13953
I20260810 07:20:34.007331 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:34.007679 1527 external_mini_cluster.cc:1428] Setting key 18306251ea5337b338057838b834cb38
Aug 10 07:20:34 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346433, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:34.014362 13921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:34.001519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:56293 (local address 127.1.125.254:40525)
0810 07:20:34.001694 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:34.001698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:34.002783 (+ 1085us) server_negotiation.cc:407] Connection header received
0810 07:20:34.003845 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:34.003847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:34.003895 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:34.003970 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:34.005252 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:34.005804 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:34.007055 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:34.007340 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:34.007465 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:34.010448 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:34.010452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:34.010455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:34.010483 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:34.012028 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:34.012649 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:34.012653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:34.012654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:34.012749 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:34.013135 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:34.013138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:34.013140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:34.013346 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:34.013460 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:34.014021 (+ 561us) server_negotiation.cc:299] Negotiation successful
0810 07:20:34.014163 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:20:34.015326 14143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:34.001919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:56293)
0810 07:20:34.002571 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:34.002662 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:34.003631 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:34.004152 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:34.004159 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:34.004595 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:34.005100 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:34.005114 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:34.005985 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:34.005987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:34.006869 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:34.006878 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:34.007548 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:34.007555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:34.007763 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:34.008458 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:34.008475 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:34.010275 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:34.012169 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:34.012176 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:34.012195 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:34.012540 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:34.012858 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:34.012861 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:34.012863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:34.013026 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:34.013460 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:34.013469 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:34.013752 (+ 283us) client_negotiation.cc:769] Sending connection context
0810 07:20:34.013950 (+ 198us) client_negotiation.cc:240] Negotiation successful
0810 07:20:34.014325 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":102,"threads_started":1}
I20260810 07:20:34.016781 14141 heartbeater.cc:347] Connected to a master server at 127.1.125.254:40525
I20260810 07:20:34.017110 14141 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:34.017789 14141 heartbeater.cc:511] Master 127.1.125.254:40525 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:20:34.019655 13869 ts_manager.cc:194] Registered new tserver with Master: 5eb02b5dcc9e4fe78cdd2e0c682f4226 (127.1.125.193:40617)
I20260810 07:20:34.020911 13869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:56293
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:20:34.038991 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:40525
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:34.164507 14148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:34.164850 14148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:34.164930 14148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:34.169625 14148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:34.169757 14148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:34.169896 14148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:34.176630 14148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:34.177912 14148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:34.179172 14148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:34.190145 14155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:34.190327 14148 server_base.cc:1034] running on GCE node
W20260810 07:20:34.190164 14156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:34.190207 14158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:34.190865 14148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:34.191529 14148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:34.192689 14148 hybrid_clock.cc:648] HybridClock initialized: now 1786346434192673 us; error 32 us; skew 500 ppm
Aug 10 07:20:34 dist-test-slave-44d2 krb5kdc[13813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346434, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:34.195393 14148 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:20:34.196609 14148 webserver.cc:460] Webserver started at http://127.1.125.194:36673/ using document root <none> and password file <none>
I20260810 07:20:34.197327 14148 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:34.197423 14148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:34.197666 14148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:34.199703 14148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "27df0f725b3d4d779650967e3aa92655"
format_stamp: "Formatted at 2026-08-10 07:20:34 on dist-test-slave-44d2"
server_key: "1046f8e02b0e0a3b3880fd45f2621fc5"
server_key_iv: "93897d684bc147ac8b81c7a1ee15fed4"
server_key_version: "encryptionkey@0"
I20260810 07:20:34.200281 14148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "27df0f725b3d4d779650967e3aa92655"
format_stamp: "Formatted at 2026-08-10 07:20:34 on dist-test-slave-44d2"
server_key: "1046f8e02b0e0a3b3880fd45f2621fc5"
server_key_iv: "93897d684bc147ac8b81c7a1ee15fed4"
server_key_version: "encryptionkey@0"
I20260810 07:20:34.204108 14148 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260810 07:20:34.206619 14169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:34.207798 14148 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:34.207942 14148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "27df0f725b3d4d779650967e3aa92655"
format_stamp: "Formatted at 2026-08-10 07:20:34 on dist-test-slave-44d2"
server_key: "1046f8e02b0e0a3b3880fd45f2621fc5"
server_key_iv: "93897d684bc147ac8b81c7a1ee15fed4"
server_key_version: "encryptionkey@0"
I20260810 07:20:34.208063 14148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:34.247637 14148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:34.250833 14148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:34.251088 14148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:34.251784 14148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:34.252835 14148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:34.252899 14148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:34.252957 14148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:34.252998 14148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:34.262960 14148 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:37521
I20260810 07:20:34.262990 14337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:37521 every 8 connection(s)
I20260810 07:20:34.264021 14148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:34.267227 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 14148
I20260810 07:20:34.267364 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:34.267735 1527 external_mini_cluster.cc:1428] Setting key 3a6cd2ca0124201112aad76fd84835ef
Aug 10 07:20:34 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346434, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
I20260810 07:20:34.279156 13921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:34.266101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:53791 (local address 127.1.125.254:40525)
0810 07:20:34.266247 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:34.266251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:34.267453 (+ 1202us) server_negotiation.cc:407] Connection header received
0810 07:20:34.268610 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:34.268613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:34.268657 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:34.268731 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:34.270283 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:34.270799 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:34.272325 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:34.272642 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:34.272760 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:34.275423 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:34.275426 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:34.275428 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:34.275452 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:34.276992 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:34.277559 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:34.277563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:34.277565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:34.277652 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:34.278006 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:34.278010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:34.278011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:34.278210 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:34.278325 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:34.278830 (+ 505us) server_negotiation.cc:299] Negotiation successful
0810 07:20:34.278965 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:20:34.280012 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:34.266494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:53791)
0810 07:20:34.267261 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:34.267329 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:34.268347 (+ 1018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:34.268907 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:34.268918 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:34.269399 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:34.270107 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:34.270132 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:34.270931 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:34.270938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:34.272183 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:34.272193 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:34.272803 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:34.272808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:34.273009 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:34.273651 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:34.273668 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:34.275239 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:34.277159 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:34.277165 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:34.277180 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:34.277447 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:34.277749 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:34.277753 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:34.277755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:34.277903 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:34.278326 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:34.278334 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:34.278587 (+ 253us) client_negotiation.cc:769] Sending connection context
0810 07:20:34.278782 (+ 195us) client_negotiation.cc:240] Negotiation successful
0810 07:20:34.279108 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":115,"threads_started":1}
I20260810 07:20:34.281291 14340 heartbeater.cc:347] Connected to a master server at 127.1.125.254:40525
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
I20260810 07:20:34.281574 14340 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:34.282230 14340 heartbeater.cc:511] Master 127.1.125.254:40525 requested a full tablet report, sending...
I20260810 07:20:34.283463 13869 ts_manager.cc:194] Registered new tserver with Master: 27df0f725b3d4d779650967e3aa92655 (127.1.125.194:37521)
I20260810 07:20:34.284077 13869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:53791
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:20:34.299935 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:40525
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:34.424950 14347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:34.425256 14347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:34.425366 14347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:34.429522 14347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:34.429620 14347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:34.429714 14347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:34.435031 14347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:34.436115 14347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:34.437218 14347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:34.447539 14355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:34.448345 14356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:34.448599 14359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:34.449393 14347 server_base.cc:1034] running on GCE node
I20260810 07:20:34.449788 14347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:34.450399 14347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:34.451583 14347 hybrid_clock.cc:648] HybridClock initialized: now 1786346434451569 us; error 33 us; skew 500 ppm
Aug 10 07:20:34 dist-test-slave-44d2 krb5kdc[13813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346434, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:34.454612 14347 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:20:34.455607 14347 webserver.cc:460] Webserver started at http://127.1.125.195:33689/ using document root <none> and password file <none>
I20260810 07:20:34.456218 14347 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:34.456281 14347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:34.456591 14347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:34.458705 14347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "4e244652be4d4312974915ab4f220024"
format_stamp: "Formatted at 2026-08-10 07:20:34 on dist-test-slave-44d2"
server_key: "5f9a1bda61658a778d28bd77b37e6261"
server_key_iv: "1c59c671e8db7124584575a78b79ef74"
server_key_version: "encryptionkey@0"
I20260810 07:20:34.459306 14347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "4e244652be4d4312974915ab4f220024"
format_stamp: "Formatted at 2026-08-10 07:20:34 on dist-test-slave-44d2"
server_key: "5f9a1bda61658a778d28bd77b37e6261"
server_key_iv: "1c59c671e8db7124584575a78b79ef74"
server_key_version: "encryptionkey@0"
I20260810 07:20:34.463279 14347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:34.465883 14370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:34.467186 14347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:20:34.467312 14347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "4e244652be4d4312974915ab4f220024"
format_stamp: "Formatted at 2026-08-10 07:20:34 on dist-test-slave-44d2"
server_key: "5f9a1bda61658a778d28bd77b37e6261"
server_key_iv: "1c59c671e8db7124584575a78b79ef74"
server_key_version: "encryptionkey@0"
I20260810 07:20:34.467422 14347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:34.481580 14347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:34.484586 14347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:34.484784 14347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:34.485381 14347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:34.486210 14347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:34.486267 14347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:34.486369 14347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:34.486406 14347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:34.496572 14347 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:38575
I20260810 07:20:34.496639 14549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:38575 every 8 connection(s)
I20260810 07:20:34.497567 14347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:34.499033 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 14347
I20260810 07:20:34.499233 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:34.499600 1527 external_mini_cluster.cc:1428] Setting key 75b031f04b4fa05da702975d9954484b
Aug 10 07:20:34 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346434, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:34.511296 13921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:34.499783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:55451 (local address 127.1.125.254:40525)
0810 07:20:34.499975 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:34.499980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:34.501209 (+ 1229us) server_negotiation.cc:407] Connection header received
0810 07:20:34.502147 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:34.502150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:34.502198 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:34.502268 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:34.503494 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:34.504013 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:34.505334 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:34.505550 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:34.505631 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:34.508129 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:34.508133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:34.508135 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:34.508159 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:34.509461 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:34.509941 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:34.509944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:34.509945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:34.510023 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:34.510332 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:34.510334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:34.510335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:34.510507 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:34.510581 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:34.511024 (+ 443us) server_negotiation.cc:299] Negotiation successful
0810 07:20:34.511121 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 07:20:34.512249 14552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:34.500246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:55451)
0810 07:20:34.501047 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:34.501102 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:34.501946 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:34.502471 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:34.502478 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:34.502843 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:34.503365 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:34.503379 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:34.504179 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:34.504182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:34.505216 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:34.505228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:34.505705 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:34.505707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:34.505855 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:34.506475 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:34.506495 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:34.507976 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:34.509581 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:34.509588 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:34.509599 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:34.509852 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:34.510112 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:34.510114 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:34.510116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:34.510252 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:34.510598 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:34.510603 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:34.510823 (+ 220us) client_negotiation.cc:769] Sending connection context
0810 07:20:34.510975 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 07:20:34.511330 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":126,"threads_started":1}
I20260810 07:20:34.513480 14550 heartbeater.cc:347] Connected to a master server at 127.1.125.254:40525
I20260810 07:20:34.513753 14550 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:34.514266 14550 heartbeater.cc:511] Master 127.1.125.254:40525 requested a full tablet report, sending...
I20260810 07:20:34.515347 13869 ts_manager.cc:194] Registered new tserver with Master: 4e244652be4d4312974915ab4f220024 (127.1.125.195:38575)
I20260810 07:20:34.515926 13869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:55451
I20260810 07:20:34.525555 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab.
Entry for principal oryx/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab.
WARNING: no policy specified for oryx/127.1.125.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab.
Entry for principal oryx/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab.
WARNING: no policy specified for oryx/127.1.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab.
Entry for principal oryx/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab.
I20260810 07:20:34.610898 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 13953
I20260810 07:20:34.633216 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 14148
I20260810 07:20:34.645952 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 14347
I20260810 07:20:34.658646 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 13828
I20260810 07:20:34.670944 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:40525
--webserver_interface=127.1.125.254
--webserver_port=36781
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:40525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:34.785185 14559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:34.785521 14559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:34.785611 14559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:34.789763 14559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:34.789841 14559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:34.789875 14559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:34.789923 14559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:34.789954 14559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:34.795469 14559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:40525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:40525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=36781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:34.796618 14559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:34.797770 14559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:34.806595 14569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:34.806587 14568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:34.806721 14559 server_base.cc:1034] running on GCE node
W20260810 07:20:34.806587 14572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:34.807477 14559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:34.808699 14559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:34.809906 14559 hybrid_clock.cc:648] HybridClock initialized: now 1786346434809890 us; error 29 us; skew 500 ppm
Aug 10 07:20:34 dist-test-slave-44d2 krb5kdc[13813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346434, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:34.812976 14559 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:20:34.813999 14559 webserver.cc:460] Webserver started at http://127.1.125.254:36781/ using document root <none> and password file <none>
I20260810 07:20:34.814584 14559 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:34.814664 14559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:34.818749 14559 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:20:34.820917 14585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:34.822263 14559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:34.822422 14559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "1b52442a4da444599fc49b69ccfcc750"
format_stamp: "Formatted at 2026-08-10 07:20:33 on dist-test-slave-44d2"
server_key: "ae7867f7ceefc7b81f7fb275741be824"
server_key_iv: "6f6e4cd69edcd61a1ef06e7bc7d92006"
server_key_version: "encryptionkey@0"
I20260810 07:20:34.822991 14559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:34.859548 14559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:34.863286 14559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:34.863523 14559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:34.871035 14559 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:40525
I20260810 07:20:34.871121 14655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:40525 every 8 connection(s)
I20260810 07:20:34.872205 14559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:34.876734 14657 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:20:34.878337 14657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750: Bootstrap starting.
I20260810 07:20:34.878782 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 14559
I20260810 07:20:34.879313 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:40617
--local_ip_for_outbound_sockets=127.1.125.193
--tserver_master_addrs=127.1.125.254:40525
--webserver_port=34895
--webserver_interface=127.1.125.193
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.125.193
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:20:34.897382 14657 log.cc:824] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:34.904227 14657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:20:34.904677 14657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750: Bootstrap complete.
I20260810 07:20:34.909055 14657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b52442a4da444599fc49b69ccfcc750" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40525 } }
I20260810 07:20:34.909476 14657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b52442a4da444599fc49b69ccfcc750, State: Initialized, Role: FOLLOWER
I20260810 07:20:34.909978 14657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b52442a4da444599fc49b69ccfcc750" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40525 } }
I20260810 07:20:34.910126 14657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:34.910214 14657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:34.910336 14657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:20:34.912216 14657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b52442a4da444599fc49b69ccfcc750" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40525 } }
I20260810 07:20:34.912626 14657 leader_election.cc:302] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b52442a4da444599fc49b69ccfcc750; no voters:
I20260810 07:20:34.913033 14657 leader_election.cc:288] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:20:34.913117 14662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:20:34.913304 14662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [term 2 LEADER]: Becoming Leader. State: Replica: 1b52442a4da444599fc49b69ccfcc750, State: Running, Role: LEADER
I20260810 07:20:34.913591 14662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b52442a4da444599fc49b69ccfcc750" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40525 } }
I20260810 07:20:34.914287 14657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:34.915112 14666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b52442a4da444599fc49b69ccfcc750. Latest consensus state: current_term: 2 leader_uuid: "1b52442a4da444599fc49b69ccfcc750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b52442a4da444599fc49b69ccfcc750" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40525 } } }
I20260810 07:20:34.915129 14664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1b52442a4da444599fc49b69ccfcc750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b52442a4da444599fc49b69ccfcc750" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40525 } } }
I20260810 07:20:34.915275 14666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:34.915275 14664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:34.916726 14668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:34.920130 14668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:34.920859 14668 catalog_manager.cc:1259] Loaded cluster ID: 44487248a315429197364e1970c82d71
I20260810 07:20:34.920924 14668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:34.923359 14668 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:34.924041 14668 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1b52442a4da444599fc49b69ccfcc750: Loaded TSK: 0
I20260810 07:20:34.925283 14668 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:20:35.001832 14660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:35.002090 14660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:35.002132 14660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:35.006382 14660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:35.006443 14660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:35.006527 14660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:35.012182 14660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.193.keytab
--principal=oryx/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:40617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=34895
--tserver_master_addrs=127.1.125.254:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:35.013329 14660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:35.014475 14660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:35.024747 14694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:35.024775 14705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:35.024786 14695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:35.025059 14660 server_base.cc:1034] running on GCE node
I20260810 07:20:35.025570 14660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:35.026343 14660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:35.027522 14660 hybrid_clock.cc:648] HybridClock initialized: now 1786346435027447 us; error 86 us; skew 500 ppm
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346435, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:35.030478 14660 init.cc:377] Logged in from keytab as oryx/127.1.125.193@KRBTEST.COM (short username oryx)
I20260810 07:20:35.031404 14660 webserver.cc:460] Webserver started at http://127.1.125.193:34895/ using document root <none> and password file <none>
I20260810 07:20:35.031975 14660 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:35.032054 14660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:35.036113 14660 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:35.038434 14717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:35.039745 14660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:20:35.039887 14660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "5eb02b5dcc9e4fe78cdd2e0c682f4226"
format_stamp: "Formatted at 2026-08-10 07:20:33 on dist-test-slave-44d2"
server_key: "321a487bc0791d99122f5212921ee112"
server_key_iv: "13f5fa31a267a1fc384cec793083edc1"
server_key_version: "encryptionkey@0"
I20260810 07:20:35.040450 14660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:35.066963 14660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:35.070961 14660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:35.071215 14660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:35.072011 14660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:35.073083 14660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:35.073167 14660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:35.073252 14660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:35.073302 14660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:35.083618 14660 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:40617
I20260810 07:20:35.083640 14881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:40617 every 8 connection(s)
I20260810 07:20:35.085007 14660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:35.087651 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 14660
I20260810 07:20:35.088115 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:37521
--local_ip_for_outbound_sockets=127.1.125.194
--tserver_master_addrs=127.1.125.254:40525
--webserver_port=36673
--webserver_interface=127.1.125.194
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.125.194
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:20:35.100224 14885 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:35.087155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:55915 (local address 127.1.125.254:40525)
0810 07:20:35.087727 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:35.087753 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:35.088454 (+ 701us) server_negotiation.cc:407] Connection header received
0810 07:20:35.089681 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:35.089693 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:35.090024 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:35.090385 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:35.091730 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.092932 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.094236 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.094757 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.094963 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.099473 (+ 4510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:55915: BlockingRecv error: recv got EOF from 127.1.125.193:55915 (error 108)
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":133,"threads_started":1}
I20260810 07:20:35.100310 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:35.087378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:55915)
0810 07:20:35.088265 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:35.088321 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:35.089233 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:35.090484 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:35.090493 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:35.090999 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:35.091528 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.091543 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.093089 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.093093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.094089 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.094100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.094957 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.094961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.095168 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.095894 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:35.095912 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:35.099202 (+ 3290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":140,"threads_started":1}
W20260810 07:20:35.100951 14882 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:40525 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.125.254:40525: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:35.107024 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:35.101407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:53431)
0810 07:20:35.101547 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:35.101562 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:35.101640 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:35.101862 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:35.101865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:35.102058 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:35.102182 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.102188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.103033 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.103037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.103922 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.103933 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.104396 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.104400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.104525 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.104992 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:35.105007 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:35.106881 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 07:20:35.107249 14885 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:35.101530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:53431 (local address 127.1.125.254:40525)
0810 07:20:35.101651 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:35.101655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:35.101668 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:20:35.101719 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:35.101721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:35.101760 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:35.101825 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:35.102307 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.102896 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.104064 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.104282 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.104373 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.107008 (+ 2635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:53431: BlockingRecv error: recv got EOF from 127.1.125.193:53431 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:35.112681 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:35.107693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:58683)
0810 07:20:35.107801 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:35.107817 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:35.107893 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:35.108112 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:35.108114 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:35.108236 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:35.108303 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.108307 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.109307 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.109310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.109812 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.109824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.110369 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.110371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.110426 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.110903 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:35.110915 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:35.112597 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 07:20:35.112843 14885 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:35.107770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:58683 (local address 127.1.125.254:40525)
0810 07:20:35.107907 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:35.107911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:35.107925 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:20:35.107960 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:35.107963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:35.108004 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:35.108086 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:35.108403 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.109204 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.109927 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.110261 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.110361 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.112740 (+ 2379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:58683: BlockingRecv error: recv got EOF from 127.1.125.193:58683 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 07:20:35.112888 14882 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:20:35.208631 14887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:35.208877 14887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:35.208948 14887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:35.212949 14887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:35.213021 14887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:35.213124 14887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:35.218789 14887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.194.keytab
--principal=oryx/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:37521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=36673
--tserver_master_addrs=127.1.125.254:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:35.219952 14887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:35.221062 14887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:35.230960 14895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:35.230968 14896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:35.231199 14887 server_base.cc:1034] running on GCE node
W20260810 07:20:35.230985 14899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:35.231688 14887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:35.232345 14887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:35.233513 14887 hybrid_clock.cc:648] HybridClock initialized: now 1786346435233500 us; error 40 us; skew 500 ppm
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346435, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:35.236393 14887 init.cc:377] Logged in from keytab as oryx/127.1.125.194@KRBTEST.COM (short username oryx)
I20260810 07:20:35.237323 14887 webserver.cc:460] Webserver started at http://127.1.125.194:36673/ using document root <none> and password file <none>
I20260810 07:20:35.237917 14887 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:35.238039 14887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:35.242106 14887 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:20:35.244374 14908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:35.245878 14887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:35.246019 14887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "27df0f725b3d4d779650967e3aa92655"
format_stamp: "Formatted at 2026-08-10 07:20:34 on dist-test-slave-44d2"
server_key: "1046f8e02b0e0a3b3880fd45f2621fc5"
server_key_iv: "93897d684bc147ac8b81c7a1ee15fed4"
server_key_version: "encryptionkey@0"
I20260810 07:20:35.246623 14887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:35.274847 14887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:35.278023 14887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:35.278252 14887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:35.278904 14887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:35.279788 14887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:35.279865 14887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:35.279942 14887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:35.279987 14887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:35.289441 14887 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:37521
I20260810 07:20:35.289476 15080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:37521 every 8 connection(s)
I20260810 07:20:35.290488 14887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:35.296455 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 14887
I20260810 07:20:35.296963 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:38575
--local_ip_for_outbound_sockets=127.1.125.195
--tserver_master_addrs=127.1.125.254:40525
--webserver_port=33689
--webserver_interface=127.1.125.195
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.125.195
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:20:35.302799 14885 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:35.292522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:43469 (local address 127.1.125.254:40525)
0810 07:20:35.292695 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:35.292699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:35.293878 (+ 1179us) server_negotiation.cc:407] Connection header received
0810 07:20:35.294738 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:35.294741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:35.294795 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:35.294874 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:35.296537 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.297200 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.298490 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.298726 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.298847 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.302606 (+ 3759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:43469: BlockingRecv error: recv got EOF from 127.1.125.194:43469 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 07:20:35.303319 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:35.292966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:43469)
0810 07:20:35.293706 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:35.293750 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:35.294548 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:35.295055 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:35.295061 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:35.295532 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:35.296322 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.296347 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.297342 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.297345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.298359 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.298369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.298894 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.298898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.299081 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.299776 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:35.299793 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:35.302400 (+ 2607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":129,"threads_started":1}
W20260810 07:20:35.303965 15084 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:40525 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.125.254:40525: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:35.310701 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:35.304517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:54773)
0810 07:20:35.304626 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:35.304641 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:35.304720 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:35.305039 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:35.305042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:35.305176 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:35.305265 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.305270 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.306389 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.306391 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.306929 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.306937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.307625 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.307629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.307715 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.308384 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:35.308396 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:35.310597 (+ 2201us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 07:20:35.311169 14885 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:35.304606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:54773 (local address 127.1.125.254:40525)
0810 07:20:35.304757 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:35.304765 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:35.304782 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:20:35.304830 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:35.304835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:35.304891 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:35.304993 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:35.305393 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.306257 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.307057 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.307474 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.307608 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.310790 (+ 3182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:54773: BlockingRecv error: recv got EOF from 127.1.125.194:54773 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:35.317095 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:35.311321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:47687)
0810 07:20:35.311616 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:35.311629 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:35.311710 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:35.312020 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:35.312023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:35.312163 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:35.312245 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.312250 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.313310 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.313313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.313826 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.313834 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.314435 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.314438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.314504 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.315018 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:35.315040 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:35.317007 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226}
W20260810 07:20:35.317287 14885 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:35.311356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:47687 (local address 127.1.125.254:40525)
0810 07:20:35.311563 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:35.311567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:35.311697 (+ 130us) server_negotiation.cc:407] Connection header received
0810 07:20:35.311796 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:35.311799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:35.311850 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:35.311935 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:35.312359 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.313192 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.313979 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.314324 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.314425 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.317165 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:47687: BlockingRecv error: recv got EOF from 127.1.125.194:47687 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
W20260810 07:20:35.317502 15084 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:20:35.422124 15088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:35.422384 15088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:35.422472 15088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:35.426716 15088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:35.426776 15088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:35.426908 15088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:35.432268 15088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/krb5kdc/oryx_127.1.125.195.keytab
--principal=oryx/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:38575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=33689
--tserver_master_addrs=127.1.125.254:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:35.433893 15088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:35.434970 15088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:35.444702 15099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:35.444783 15103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:35.444913 15088 server_base.cc:1034] running on GCE node
W20260810 07:20:35.444943 15100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:35.445359 15088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:35.446025 15088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:35.447218 15088 hybrid_clock.cc:648] HybridClock initialized: now 1786346435447183 us; error 47 us; skew 500 ppm
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346435, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:35.450047 15088 init.cc:377] Logged in from keytab as oryx/127.1.125.195@KRBTEST.COM (short username oryx)
I20260810 07:20:35.450915 15088 webserver.cc:460] Webserver started at http://127.1.125.195:33689/ using document root <none> and password file <none>
I20260810 07:20:35.451498 15088 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:35.451609 15088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:35.455749 15088 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260810 07:20:35.457933 15113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:35.459164 15088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:20:35.459313 15088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "4e244652be4d4312974915ab4f220024"
format_stamp: "Formatted at 2026-08-10 07:20:34 on dist-test-slave-44d2"
server_key: "5f9a1bda61658a778d28bd77b37e6261"
server_key_iv: "1c59c671e8db7124584575a78b79ef74"
server_key_version: "encryptionkey@0"
I20260810 07:20:35.459875 15088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:35.491778 15088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:35.495252 15088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:35.495508 15088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:35.496186 15088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:35.497116 15088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:35.497187 15088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:35.497259 15088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:35.497319 15088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:35.506871 15088 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:38575
I20260810 07:20:35.506923 15269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:38575 every 8 connection(s)
I20260810 07:20:35.507908 15088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:35.516060 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 15088
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:20:35.519682 14885 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:35.509756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:35845 (local address 127.1.125.254:40525)
0810 07:20:35.509901 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:35.509905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:35.510931 (+ 1026us) server_negotiation.cc:407] Connection header received
0810 07:20:35.511853 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:35.511857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:35.511920 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:35.512027 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:35.513269 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.513810 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.515044 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.515304 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.515418 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.519570 (+ 4152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:35845: BlockingRecv error: recv got EOF from 127.1.125.195:35845 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:20:35.520424 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:35.510100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:35845)
0810 07:20:35.510764 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:35.510807 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:35.511634 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:35.512222 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:35.512229 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:35.512620 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:35.513111 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.513126 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.513949 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.513958 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.514907 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.514917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.515475 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.515480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.515687 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.516417 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:35.516434 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:35.519460 (+ 3026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":116,"threads_started":1}
W20260810 07:20:35.520962 15270 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:40525 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.125.254:40525: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:35.526108 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:35.517204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41388 (local address 127.1.125.254:40525)
0810 07:20:35.517332 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:35.517335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:35.517350 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:20:35.517400 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:35.517401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:35.517437 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:35.517503 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:35.518466 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.519052 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.520229 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.520450 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.520612 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.522138 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:35.522144 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:35.522159 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:35.522186 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:35.523926 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:35.524378 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:35.524391 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:35.524396 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:35.524579 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:35.524865 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:35.524869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:35.524871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:35.525259 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:35.525467 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:35.525751 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 07:20:35.525992 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:20:35.526396 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:35.521464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:48497)
0810 07:20:35.521560 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:35.521575 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:35.521650 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:35.521891 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:35.521894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:35.522052 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:35.522152 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.522157 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.522871 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.522875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.523415 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.523424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.523874 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.523877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.523979 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.524422 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:35.524434 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:35.526270 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 07:20:35.526826 14885 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:35.521568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:48497 (local address 127.1.125.254:40525)
0810 07:20:35.521688 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:35.521694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:35.521709 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:20:35.521744 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:35.521746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:35.521788 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:35.521872 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:35.522272 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.522759 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.523544 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.523762 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.523870 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.526691 (+ 2821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:48497: BlockingRecv error: recv got EOF from 127.1.125.195:48497 (error 108)
Metrics: {"server-negotiator.queue_time_us":35,"spinlock_wait_cycles":768}
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:35 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:35.532122 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:35.526879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:56625)
0810 07:20:35.527176 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:35.527192 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:35.527268 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:35.527511 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:35.527513 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:35.527635 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:35.527748 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.527754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.528670 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.528674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.529255 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:35.529264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.529729 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.529732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:35.529811 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.530274 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:35.530287 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:35.531970 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227}
W20260810 07:20:35.532209 15089 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:35.527158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:56625 (local address 127.1.125.254:40525)
0810 07:20:35.527304 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:35.527308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:35.527340 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:20:35.527372 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:35.527374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:35.527412 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:35.527494 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:35.527863 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.528552 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.529375 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:35.529615 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:35.529724 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:35.532095 (+ 2371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:56625: BlockingRecv error: recv got EOF from 127.1.125.195:56625 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260810 07:20:35.532392 15270 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 07:20:36 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:36 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:36.119333 15280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:36.113523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:49371)
0810 07:20:36.113816 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:36.113830 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:36.113984 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:36.114245 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:36.114247 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:36.114436 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:36.114560 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:36.114566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:36.115350 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:36.115353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:36.116003 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:36.116014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:36.116611 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:36.116615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:36.116692 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:36.117255 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:36.117267 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:36.119206 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":110,"threads_started":1}
W20260810 07:20:36.119522 15282 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:36.113627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:49371 (local address 127.1.125.254:40525)
0810 07:20:36.113876 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:36.113879 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:36.113920 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:20:36.114068 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:36.114070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:36.114118 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:36.114201 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:36.114692 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:36.115249 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:36.116135 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:36.116460 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:36.116891 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:36.119345 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:49371: BlockingRecv error: recv got EOF from 127.1.125.193:49371 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":89,"threads_started":1}
Aug 10 07:20:36 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:36 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:36.323827 15283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:36.318253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:43727)
0810 07:20:36.318546 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:36.318565 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:36.318689 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:36.318911 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:36.318914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:36.319121 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:36.319222 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:36.319228 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:36.319962 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:36.319965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:36.320640 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:36.320651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:36.321066 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:36.321069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:36.321145 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:36.321747 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:36.321761 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:36.323697 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":102,"threads_started":1}
W20260810 07:20:36.323987 15282 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:36.318308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:43727 (local address 127.1.125.254:40525)
0810 07:20:36.318467 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:36.318471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:36.318659 (+ 188us) server_negotiation.cc:407] Connection header received
0810 07:20:36.318771 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:36.318774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:36.318819 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:36.318896 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:36.319328 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:36.319860 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:36.320751 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:36.320964 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:36.321044 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:36.323826 (+ 2782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:43727: BlockingRecv error: recv got EOF from 127.1.125.194:43727 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 07:20:36 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:36 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:36.538983 15285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:36.533118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:40839)
0810 07:20:36.533453 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:36.533467 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:36.533631 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:36.533940 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:36.533944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:36.534176 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:36.534306 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:36.534312 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:36.535145 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:36.535150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:36.535749 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:36.535759 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:36.536275 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:36.536278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:36.536382 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:36.536966 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:36.536978 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:36.538800 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":99,"threads_started":1}
W20260810 07:20:36.539040 15282 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:36.533191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:40839 (local address 127.1.125.254:40525)
0810 07:20:36.533380 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:36.533385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:36.533548 (+ 163us) server_negotiation.cc:407] Connection header received
0810 07:20:36.533743 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:36.533747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:36.533808 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:36.533907 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:36.534439 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:36.534991 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:36.535881 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:36.536132 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:36.536258 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:36.538930 (+ 2672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:40839: BlockingRecv error: recv got EOF from 127.1.125.195:40839 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 10 07:20:37 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:37 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:37.126192 15286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:37.120149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:33451)
0810 07:20:37.120460 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:37.120475 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:37.120662 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:37.120944 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:37.120947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:37.121140 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:37.121242 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:37.121248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:37.122007 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:37.122009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:37.122811 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:37.122824 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:37.123409 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:37.123412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:37.123528 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:37.124088 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:37.124100 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:37.126034 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":89,"threads_started":1}
W20260810 07:20:37.126319 15287 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:37.120278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:33451 (local address 127.1.125.254:40525)
0810 07:20:37.120539 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:37.120550 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:37.120611 (+ 61us) server_negotiation.cc:407] Connection header received
0810 07:20:37.120770 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:37.120772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:37.120826 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:37.120905 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:37.121350 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:37.121910 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:37.122940 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:37.123270 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:37.123392 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:37.126170 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:33451: BlockingRecv error: recv got EOF from 127.1.125.193:33451 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":59,"threads_started":1}
Aug 10 07:20:37 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:37 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:37.330296 15288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:37.324675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:34733)
0810 07:20:37.324985 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:37.324999 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:37.325122 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:37.325353 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:37.325356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:37.325563 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:37.325669 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:37.325675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:37.326511 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:37.326514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:37.327128 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:37.327138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:37.327577 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:37.327580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:37.327643 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:37.328147 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:37.328159 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:37.330148 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":128,"threads_started":1}
W20260810 07:20:37.330487 15287 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:37.324770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:34733 (local address 127.1.125.254:40525)
0810 07:20:37.324918 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:37.324921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:37.325050 (+ 129us) server_negotiation.cc:407] Connection header received
0810 07:20:37.325193 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:37.325195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:37.325239 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:37.325308 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:37.325774 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:37.326369 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:37.327236 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:37.327457 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:37.327585 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:37.330337 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:34733: BlockingRecv error: recv got EOF from 127.1.125.194:34733 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 07:20:37 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:37 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:37.546113 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:37.539878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:39299)
0810 07:20:37.540239 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:37.540257 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:37.540412 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:37.540724 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:37.540727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:37.540941 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:37.541060 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:37.541067 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:37.541860 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:37.541865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:37.542436 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:37.542444 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:37.543100 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:37.543109 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:37.543180 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:37.543795 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:37.543812 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:37.545899 (+ 2087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":135,"threads_started":1}
W20260810 07:20:37.546130 15287 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:37.540085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:39299 (local address 127.1.125.254:40525)
0810 07:20:37.540239 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:37.540246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:37.540343 (+ 97us) server_negotiation.cc:407] Connection header received
0810 07:20:37.540542 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:37.540545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:37.540605 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:37.540696 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:37.541182 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:37.541743 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:37.542567 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:37.542843 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:37.542946 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:37.546032 (+ 3086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:39299: BlockingRecv error: recv got EOF from 127.1.125.195:39299 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 07:20:38 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:38 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:38.132808 15290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:38.126953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:58265)
0810 07:20:38.127313 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:38.127344 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:38.127440 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:38.127729 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:38.127731 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:38.127922 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:38.128032 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:38.128038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:38.128820 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:38.128823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:38.129371 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:38.129378 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:38.130034 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:38.130038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:38.130168 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:38.130731 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:38.130744 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:38.132626 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":93,"threads_started":1}
W20260810 07:20:38.132887 15291 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:38.127076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:58265 (local address 127.1.125.254:40525)
0810 07:20:38.127408 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:38.127414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:38.127433 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:20:38.127538 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:38.127540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:38.127608 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:38.127686 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:38.128143 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:38.128710 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:38.129492 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:38.129864 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:38.129971 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:38.132761 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:58265: BlockingRecv error: recv got EOF from 127.1.125.193:58265 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":108,"threads_started":1}
Aug 10 07:20:38 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:38 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:38.336860 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:38.331248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:44617)
0810 07:20:38.331540 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:38.331556 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:38.331667 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:38.331859 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:38.331861 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:38.332055 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:38.332163 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:38.332169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:38.332965 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:38.332968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:38.333550 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:38.333558 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:38.333973 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:38.333976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:38.334041 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:38.334567 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:38.334579 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:38.336727 (+ 2148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":111,"threads_started":1}
W20260810 07:20:38.337113 15291 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:38.331308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:44617 (local address 127.1.125.254:40525)
0810 07:20:38.331461 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:38.331465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:38.331619 (+ 154us) server_negotiation.cc:407] Connection header received
0810 07:20:38.331735 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:38.331737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:38.331780 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:38.331847 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:38.332269 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:38.332853 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:38.333655 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:38.333873 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:38.333953 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:38.336946 (+ 2993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:44617: BlockingRecv error: recv got EOF from 127.1.125.194:44617 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 07:20:38 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:38 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:38.552939 15294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:38.547118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:58199)
0810 07:20:38.547412 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:38.547425 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:38.547562 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:38.547865 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:38.547869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:38.548096 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:38.548197 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:38.548202 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:38.549085 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:38.549089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:38.549679 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:38.549688 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:38.550186 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:38.550190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:38.550283 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:38.550777 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:38.550789 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:38.552791 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":100,"threads_started":1}
W20260810 07:20:38.553068 15291 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:38.547444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:58199 (local address 127.1.125.254:40525)
0810 07:20:38.547627 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:38.547632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:38.547648 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:20:38.547687 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:38.547690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:38.547744 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:38.547836 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:38.548342 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:38.548960 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:38.549808 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:38.550054 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:38.550179 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:38.552938 (+ 2759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:58199: BlockingRecv error: recv got EOF from 127.1.125.195:58199 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 10 07:20:39 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:39 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:39.145478 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:39.133712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:40541)
0810 07:20:39.134240 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:39.134257 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:39.134369 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:39.134639 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:39.134641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:39.134895 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:39.135001 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:39.135008 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:39.135799 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:39.135801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:39.136556 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:39.136567 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:39.137051 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:39.137054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:39.137124 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:39.142838 (+ 5714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:39.142903 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:39.145346 (+ 2443us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":131,"threads_started":1}
W20260810 07:20:39.145707 15296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:39.133940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:40541 (local address 127.1.125.254:40525)
0810 07:20:39.134240 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:39.134244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:39.134339 (+ 95us) server_negotiation.cc:407] Connection header received
0810 07:20:39.134467 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:39.134469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:39.134516 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:39.134598 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:39.135125 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:39.135694 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:39.136670 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:39.136907 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:39.136998 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:39.145541 (+ 8543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:40541: BlockingRecv error: recv got EOF from 127.1.125.193:40541 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":102,"threads_started":1}
Aug 10 07:20:39 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:39 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:39.343721 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:39.337880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:33375)
0810 07:20:39.338196 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:39.338211 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:39.338329 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:39.338590 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:39.338592 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:39.338788 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:39.338892 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:39.338897 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:39.339738 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:39.339741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:39.340299 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:39.340307 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:39.340738 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:39.340740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:39.340807 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:39.341413 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:39.341426 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:39.343538 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":79,"threads_started":1}
W20260810 07:20:39.343921 15296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:39.337971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:33375 (local address 127.1.125.254:40525)
0810 07:20:39.338212 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:39.338218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:39.338275 (+ 57us) server_negotiation.cc:407] Connection header received
0810 07:20:39.338423 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:39.338426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:39.338475 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:39.338546 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:39.338995 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:39.339638 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:39.340400 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:39.340637 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:39.340716 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:39.343691 (+ 2975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:33375: BlockingRecv error: recv got EOF from 127.1.125.194:33375 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
Aug 10 07:20:39 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:39 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:39.559669 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:39.553934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:57421)
0810 07:20:39.554209 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:39.554232 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:39.554339 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:39.554610 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:39.554613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:39.554810 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:39.554918 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:39.554924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:39.555685 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:39.555688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:39.556271 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:39.556279 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:39.556863 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:39.556868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:39.557006 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:39.557534 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:39.557547 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:39.559530 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":91,"threads_started":1}
W20260810 07:20:39.559747 15296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:39.553991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:57421 (local address 127.1.125.254:40525)
0810 07:20:39.554130 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:39.554134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:39.554300 (+ 166us) server_negotiation.cc:407] Connection header received
0810 07:20:39.554426 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:39.554429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:39.554480 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:39.554544 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:39.555037 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:39.555575 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:39.556372 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:39.556703 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:39.556803 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:39.559652 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:57421: BlockingRecv error: recv got EOF from 127.1.125.195:57421 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 07:20:40 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:40 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:40.152511 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:40.146356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:52193)
0810 07:20:40.146724 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:40.146739 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:40.146880 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:40.147282 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:40.147285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:40.147596 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:40.147732 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:40.147739 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:40.148613 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:40.148617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:40.149293 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:40.149303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:40.149802 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:40.149808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:40.149966 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:40.150496 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:40.150510 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:40.152370 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":107,"threads_started":1}
W20260810 07:20:40.152750 15302 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:40.146490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:52193 (local address 127.1.125.254:40525)
0810 07:20:40.146756 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:40.146761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:40.146843 (+ 82us) server_negotiation.cc:407] Connection header received
0810 07:20:40.147042 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:40.147047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:40.147113 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:40.147248 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:40.147902 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:40.148506 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:40.149406 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:40.149645 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:40.149775 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:40.152599 (+ 2824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:52193: BlockingRecv error: recv got EOF from 127.1.125.193:52193 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
Aug 10 07:20:40 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:40 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:40.350342 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:40.344723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:45917)
0810 07:20:40.345111 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:40.345128 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:40.345243 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:40.345578 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:40.345582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:40.345836 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:40.345957 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:40.345965 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:40.346700 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:40.346703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:40.347258 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:40.347266 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:40.347669 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:40.347672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:40.347731 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:40.348238 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:40.348250 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:40.350136 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":141,"threads_started":1}
W20260810 07:20:40.350358 15302 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:40.344902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:45917 (local address 127.1.125.254:40525)
0810 07:20:40.345098 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:40.345103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:40.345188 (+ 85us) server_negotiation.cc:407] Connection header received
0810 07:20:40.345352 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:40.345356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:40.345427 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:40.345517 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:40.346068 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:40.346607 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:40.347356 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:40.347562 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:40.347649 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:40.350260 (+ 2611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:45917: BlockingRecv error: recv got EOF from 127.1.125.194:45917 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 10 07:20:40 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:40 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:40.570984 15306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:40.560701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:40255)
0810 07:20:40.560987 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:40.561001 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:40.561101 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:40.561332 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:40.561334 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:40.561530 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:40.561646 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:40.561651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:40.562348 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:40.562350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:40.562936 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:40.562944 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:40.563379 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:40.563382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:40.563448 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:40.569019 (+ 5571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:40.569032 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:40.570858 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":93,"threads_started":1}
W20260810 07:20:40.571136 15302 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:40.560805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:40255 (local address 127.1.125.254:40525)
0810 07:20:40.560989 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:40.560994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:40.561070 (+ 76us) server_negotiation.cc:407] Connection header received
0810 07:20:40.561175 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:40.561177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:40.561226 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:40.561294 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:40.561741 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:40.562258 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:40.563048 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:40.563274 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:40.563352 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:40.570998 (+ 7646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:40255: BlockingRecv error: recv got EOF from 127.1.125.195:40255 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 10 07:20:41 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:41 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:41.159053 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:41.153352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:35555)
0810 07:20:41.153638 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:41.153654 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:41.153747 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:41.154008 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:41.154010 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:41.154207 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:41.154309 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:41.154314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:41.155071 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:41.155074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:41.155628 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:41.155637 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:41.156061 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:41.156063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:41.156123 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:41.156731 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:41.156744 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:41.158876 (+ 2132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":110,"threads_started":1}
W20260810 07:20:41.159155 15308 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:41.153455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:35555 (local address 127.1.125.254:40525)
0810 07:20:41.153711 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:41.153715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:41.153730 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:20:41.153830 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:41.153832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:41.153878 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:41.153962 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:41.154417 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:41.154960 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:41.155738 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:41.155960 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:41.156040 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:41.159004 (+ 2964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:35555: BlockingRecv error: recv got EOF from 127.1.125.193:35555 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":109,"threads_started":1}
Aug 10 07:20:41 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:41 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:41.357540 15310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:41.351122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:43461)
0810 07:20:41.351391 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:41.351410 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:41.351528 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:41.351772 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:41.351774 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:41.351969 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:41.352069 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:41.352075 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:41.353069 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:41.353071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:41.353832 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:41.353845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:41.354514 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:41.354518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:41.354617 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:41.355277 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:41.355292 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:41.357324 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":94,"threads_started":1}
W20260810 07:20:41.357592 15308 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:41.351193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:43461 (local address 127.1.125.254:40525)
0810 07:20:41.351351 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:41.351355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:41.351492 (+ 137us) server_negotiation.cc:407] Connection header received
0810 07:20:41.351608 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:41.351610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:41.351657 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:41.351729 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:41.352172 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:41.352964 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:41.354138 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:41.354380 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:41.354507 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:41.357472 (+ 2965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:43461: BlockingRecv error: recv got EOF from 127.1.125.194:43461 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 07:20:41 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:41 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:41.577433 15312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:41.571729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:60017)
0810 07:20:41.572003 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:41.572017 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:41.572128 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:41.572368 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:41.572370 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:41.572604 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:41.572708 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:41.572714 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:41.573443 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:41.573445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:41.574013 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:41.574021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:41.574419 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:41.574422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:41.574484 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:41.575084 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:41.575097 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:41.577251 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":75,"threads_started":1}
W20260810 07:20:41.577519 15308 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:41.571815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:60017 (local address 127.1.125.254:40525)
0810 07:20:41.571998 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:41.572004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:41.572089 (+ 85us) server_negotiation.cc:407] Connection header received
0810 07:20:41.572213 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:41.572215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:41.572261 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:41.572329 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:41.572811 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:41.573342 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:41.574108 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:41.574322 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:41.574399 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:41.577384 (+ 2985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:60017: BlockingRecv error: recv got EOF from 127.1.125.195:60017 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 10 07:20:42 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:42 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:42.165488 15315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:42.159948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:59507)
0810 07:20:42.160303 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:42.160318 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:42.160413 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:42.160682 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:42.160684 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:42.160875 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:42.160973 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:42.160978 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:42.161788 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:42.161790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:42.162363 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:42.162371 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:42.162795 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:42.162798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:42.162911 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:42.163432 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:42.163444 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:42.165370 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":94,"threads_started":1}
W20260810 07:20:42.165661 15313 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:42.160010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:59507 (local address 127.1.125.254:40525)
0810 07:20:42.160356 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:42.160361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:42.160374 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:20:42.160519 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:42.160522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:42.160569 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:42.160648 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:42.161113 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:42.161689 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:42.162465 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:42.162700 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:42.162778 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:42.165512 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:59507: BlockingRecv error: recv got EOF from 127.1.125.193:59507 (error 108)
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":94,"threads_started":1}
Aug 10 07:20:42 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:42 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:20:42.363966 15313 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:42.358506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:41685 (local address 127.1.125.254:40525)
0810 07:20:42.358675 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:42.358680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:42.358862 (+ 182us) server_negotiation.cc:407] Connection header received
0810 07:20:42.358983 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:42.358985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:42.359032 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:42.359108 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:42.359557 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:42.360089 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:42.360906 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:42.361130 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:42.361206 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:42.363867 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:41685: BlockingRecv error: recv got EOF from 127.1.125.194:41685 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 07:20:42.364064 15316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:42.358396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:41685)
0810 07:20:42.358750 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:42.358765 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:42.358895 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:42.359163 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:42.359165 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:42.359356 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:42.359455 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:42.359460 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:42.360183 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:42.360186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:42.360805 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:42.360815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:42.361225 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:42.361228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:42.361291 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:42.361795 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:42.361806 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:42.363727 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":132,"threads_started":1}
Aug 10 07:20:42 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:42 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:42.584038 15317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:42.578418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:56491)
0810 07:20:42.578795 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:42.578808 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:42.578951 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:42.579260 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:42.579263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:42.579488 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:42.579593 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:42.579599 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:42.580382 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:42.580387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:42.581000 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:42.581009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:42.581477 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:42.581481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:42.581568 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:42.582081 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:42.582095 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:42.583919 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":97,"threads_started":1}
W20260810 07:20:42.584254 15313 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:42.578525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:56491 (local address 127.1.125.254:40525)
0810 07:20:42.578707 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:42.578714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:42.578890 (+ 176us) server_negotiation.cc:407] Connection header received
0810 07:20:42.579056 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:42.579061 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:42.579134 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:42.579225 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:42.579720 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:42.580269 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:42.581134 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:42.581361 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:42.581471 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:42.584110 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:56491: BlockingRecv error: recv got EOF from 127.1.125.195:56491 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 07:20:43 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:43 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:43.172050 15318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:43.166373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:45763)
0810 07:20:43.166682 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:43.166696 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:43.166794 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:43.167131 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:43.167133 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:43.167322 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:43.167428 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:43.167433 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:43.168288 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:43.168290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:43.168929 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:43.168940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:43.169358 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:43.169360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:43.169425 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:43.169913 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:43.169925 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:43.171882 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":112,"threads_started":1}
W20260810 07:20:43.172199 15319 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:43.166518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:45763 (local address 127.1.125.254:40525)
0810 07:20:43.166803 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:43.166807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:43.166824 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:20:43.166933 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:43.166935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:43.166988 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:43.167092 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:43.167609 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:43.168179 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:43.169043 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:43.169261 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:43.169339 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:43.172042 (+ 2703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:45763: BlockingRecv error: recv got EOF from 127.1.125.193:45763 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":109,"threads_started":1}
Aug 10 07:20:43 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:43 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:43.371052 15321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:43.365043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:58615)
0810 07:20:43.365351 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:43.365365 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:43.365472 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:43.365746 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:43.365748 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:43.365934 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:43.366035 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:43.366040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:43.367071 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:43.367073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:43.367638 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:43.367645 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:43.368111 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:43.368114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:43.368174 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:43.368779 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:43.368794 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:43.370929 (+ 2135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":113,"threads_started":1}
W20260810 07:20:43.371304 15319 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:43.365093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:58615 (local address 127.1.125.254:40525)
0810 07:20:43.365266 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:43.365273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:43.365449 (+ 176us) server_negotiation.cc:407] Connection header received
0810 07:20:43.365581 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:43.365583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:43.365639 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:43.365717 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:43.366151 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:43.366963 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:43.367770 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:43.368004 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:43.368094 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:43.371147 (+ 3053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:58615: BlockingRecv error: recv got EOF from 127.1.125.194:58615 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 07:20:43 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:43 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:43.590219 15322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:43.584923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:35973)
0810 07:20:43.585202 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:43.585217 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:43.585334 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:43.585584 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:43.585587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:43.585773 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:43.585877 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:43.585883 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:43.586607 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:43.586610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:43.587166 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:43.587174 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:43.587643 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:43.587646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:43.587708 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:43.588248 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:43.588268 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:43.590097 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":94,"threads_started":1}
W20260810 07:20:43.590315 15319 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:43.585019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:35973 (local address 127.1.125.254:40525)
0810 07:20:43.585153 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:43.585156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:43.585289 (+ 133us) server_negotiation.cc:407] Connection header received
0810 07:20:43.585420 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:43.585422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:43.585469 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:43.585543 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:43.585972 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:43.586512 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:43.587297 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:43.587530 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:43.587634 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:43.590220 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:35973: BlockingRecv error: recv got EOF from 127.1.125.195:35973 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 07:20:44 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:44 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:44.184159 15325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:44.173089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:39525)
0810 07:20:44.173381 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:44.173399 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:44.173503 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:44.173718 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:44.173720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:44.173922 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:44.174021 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:44.174026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:44.174773 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:44.174776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:44.175399 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:44.175408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:44.175830 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:44.175832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:44.175936 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:44.181531 (+ 5595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:44.181550 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:44.183923 (+ 2373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":78,"threads_started":1}
W20260810 07:20:44.184291 15326 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:44.173196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:39525 (local address 127.1.125.254:40525)
0810 07:20:44.173440 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:44.173444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:44.173457 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:20:44.173574 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:44.173576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:44.173619 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:44.173694 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:44.174132 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:44.174661 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:44.175505 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:44.175728 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:44.175805 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:44.184118 (+ 8313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:39525: BlockingRecv error: recv got EOF from 127.1.125.193:39525 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":85,"threads_started":1}
Aug 10 07:20:44 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:44 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:44.377501 15327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:44.371823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:34231)
0810 07:20:44.372146 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:44.372160 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:44.372273 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:44.372530 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:44.372532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:44.372752 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:44.372854 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:44.372860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:44.373594 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:44.373596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:44.374147 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:44.374154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:44.374567 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:44.374571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:44.374647 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:44.375163 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:44.375174 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:44.377332 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":126,"threads_started":1}
W20260810 07:20:44.377645 15326 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:44.371915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:34231 (local address 127.1.125.254:40525)
0810 07:20:44.372068 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:44.372072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:44.372237 (+ 165us) server_negotiation.cc:407] Connection header received
0810 07:20:44.372352 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:44.372355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:44.372401 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:44.372474 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:44.372956 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:44.373473 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:44.374244 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:44.374457 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:44.374548 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:44.377476 (+ 2928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:34231: BlockingRecv error: recv got EOF from 127.1.125.194:34231 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 07:20:44 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:44 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:44.596722 15328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:44.591117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:45545)
0810 07:20:44.591408 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:44.591422 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:44.591528 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:44.591764 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:44.591767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:44.591956 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:44.592057 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:44.592062 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:44.592822 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:44.592824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:44.593378 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:44.593387 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:44.593894 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:44.593897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:44.593959 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:44.594457 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:44.594469 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:44.596593 (+ 2124us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":104,"threads_started":1}
W20260810 07:20:44.596892 15326 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:44.591173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:45545 (local address 127.1.125.254:40525)
0810 07:20:44.591319 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:44.591324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:44.591500 (+ 176us) server_negotiation.cc:407] Connection header received
0810 07:20:44.591608 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:44.591611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:44.591659 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:44.591727 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:44.592155 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:44.592717 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:44.593504 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:44.593793 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:44.593876 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:44.596732 (+ 2856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:45545: BlockingRecv error: recv got EOF from 127.1.125.195:45545 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 07:20:45 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:45 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:45.191073 15330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:45.185043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:46637)
0810 07:20:45.185367 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:45.185381 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:45.185472 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:45.185716 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:45.185718 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:45.185909 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:45.186014 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:45.186019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:45.186762 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:45.186765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:45.187332 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:45.187341 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:45.187819 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:45.187823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:45.187966 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:45.188659 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:45.188681 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:45.190938 (+ 2257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":113,"threads_started":1}
W20260810 07:20:45.191241 15331 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:45.185149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:46637 (local address 127.1.125.254:40525)
0810 07:20:45.185414 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:45.185418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:45.185432 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:20:45.185554 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:45.185556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:45.185601 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:45.185680 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:45.186129 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:45.186660 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:45.187437 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:45.187660 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:45.187737 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:45.191092 (+ 3355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:46637: BlockingRecv error: recv got EOF from 127.1.125.193:46637 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":65,"threads_started":1}
Aug 10 07:20:45 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:45 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:45.385094 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:45.378586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:57861)
0810 07:20:45.378960 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:45.378985 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:45.379101 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:45.379388 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:45.379391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:45.379604 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:45.379722 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:45.379728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:45.380684 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:45.380690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:45.381574 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:45.381592 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:45.382063 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:45.382066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:45.382132 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:45.382839 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:45.382852 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:45.384963 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":100,"threads_started":1}
W20260810 07:20:45.385253 15331 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:45.378694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:57861 (local address 127.1.125.254:40525)
0810 07:20:45.378844 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:45.378848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:45.379059 (+ 211us) server_negotiation.cc:407] Connection header received
0810 07:20:45.379185 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:45.379188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:45.379235 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:45.379311 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:45.379842 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:45.380557 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:45.381712 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:45.381954 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:45.382049 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:45.385097 (+ 3048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:57861: BlockingRecv error: recv got EOF from 127.1.125.194:57861 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 07:20:45 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:45 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:45.603382 15336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:45.597692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:57423)
0810 07:20:45.598023 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:45.598038 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:45.598129 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:45.598427 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:45.598430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:45.598627 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:45.598736 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:45.598741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:45.599563 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:45.599566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:45.600140 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:45.600148 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:45.600656 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:45.600659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:45.600726 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:45.601217 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:45.601229 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:45.603197 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":128,"threads_started":1}
W20260810 07:20:45.603444 15331 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:45.597853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:57423 (local address 127.1.125.254:40525)
0810 07:20:45.598110 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:45.598113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:45.598130 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:20:45.598219 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:45.598221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:45.598268 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:45.598346 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:45.598861 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:45.599441 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:45.600252 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:45.600545 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:45.600644 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:45.603330 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:57423: BlockingRecv error: recv got EOF from 127.1.125.195:57423 (error 108)
Metrics: {"server-negotiator.queue_time_us":171}
Aug 10 07:20:46 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:46 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:46.197782 15337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:46.192083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:36187)
0810 07:20:46.192419 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:46.192434 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:46.192580 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:46.192882 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:46.192884 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:46.193082 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:46.193189 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:46.193195 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:46.193930 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:46.193932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:46.194481 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:46.194488 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:46.194905 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:46.194908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:46.194974 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:46.195554 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:46.195567 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:46.197600 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":97,"threads_started":1}
W20260810 07:20:46.197872 15338 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:46.192152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:36187 (local address 127.1.125.254:40525)
0810 07:20:46.192453 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:46.192457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:46.192516 (+ 59us) server_negotiation.cc:407] Connection header received
0810 07:20:46.192683 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:46.192686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:46.192732 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:46.192812 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:46.193300 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:46.193834 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:46.194583 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:46.194806 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:46.194882 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:46.197736 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:36187: BlockingRecv error: recv got EOF from 127.1.125.193:36187 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":111,"threads_started":1}
Aug 10 07:20:46 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:46 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:46.391706 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:46.385976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:46069)
0810 07:20:46.386439 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:46.386475 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:46.386602 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:46.386905 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:46.386909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:46.387175 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:46.387303 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:46.387310 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:46.388126 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:46.388128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:46.388765 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:46.388774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:46.389221 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:46.389223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:46.389288 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:46.389770 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:46.389782 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:46.391586 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":95,"threads_started":1}
W20260810 07:20:46.391849 15338 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:46.386087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:46069 (local address 127.1.125.254:40525)
0810 07:20:46.386262 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:46.386266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:46.386548 (+ 282us) server_negotiation.cc:407] Connection header received
0810 07:20:46.386719 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:46.386723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:46.386779 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:46.386875 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:46.387466 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:46.388027 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:46.388878 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:46.389121 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:46.389200 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:46.391730 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:46069: BlockingRecv error: recv got EOF from 127.1.125.194:46069 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 07:20:46 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:46 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:46.610204 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:46.604251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:57303)
0810 07:20:46.604632 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:46.604648 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:46.604796 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:46.605156 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:46.605159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:46.605392 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:46.605489 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:46.605495 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:46.606282 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:46.606287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:46.606909 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:46.606918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:46.607398 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:46.607401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:46.607496 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:46.608067 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:46.608079 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:46.610008 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":95,"threads_started":1}
W20260810 07:20:46.610242 15338 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:46.604402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:57303 (local address 127.1.125.254:40525)
0810 07:20:46.604605 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:46.604611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:46.604739 (+ 128us) server_negotiation.cc:407] Connection header received
0810 07:20:46.604981 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:46.604983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:46.605032 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:46.605126 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:46.605621 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:46.606170 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:46.607035 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:46.607264 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:46.607374 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:46.610132 (+ 2758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:57303: BlockingRecv error: recv got EOF from 127.1.125.195:57303 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 10 07:20:47 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:47 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:47.204439 15343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:47.198746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:54949)
0810 07:20:47.199071 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:47.199089 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:47.199196 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:47.199461 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:47.199463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:47.199666 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:47.199772 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:47.199777 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:47.200617 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:47.200621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:47.201365 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:47.201377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:47.201854 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:47.201857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:47.201947 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:47.202449 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:47.202461 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:47.204318 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":96,"threads_started":1}
W20260810 07:20:47.204779 15344 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:47.198895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:54949 (local address 127.1.125.254:40525)
0810 07:20:47.199164 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:47.199168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:47.199184 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:20:47.199294 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:47.199297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:47.199344 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:47.199429 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:47.199889 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:47.200456 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:47.201482 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:47.201732 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:47.201845 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:47.204458 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:54949: BlockingRecv error: recv got EOF from 127.1.125.193:54949 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":67,"threads_started":1}
Aug 10 07:20:47 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:47 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:47.397904 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:47.392560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:57739)
0810 07:20:47.392849 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:47.392863 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:47.392968 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:47.393199 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:47.393201 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:47.393397 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:47.393503 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:47.393508 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:47.394238 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:47.394240 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:47.394793 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:47.394801 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:47.395223 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:47.395226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:47.395293 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:47.395863 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:47.395876 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:47.397783 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":89,"threads_started":1}
W20260810 07:20:47.398042 15344 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:47.392617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:57739 (local address 127.1.125.254:40525)
0810 07:20:47.392754 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:47.392758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:47.392928 (+ 170us) server_negotiation.cc:407] Connection header received
0810 07:20:47.393039 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:47.393041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:47.393085 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:47.393156 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:47.393599 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:47.394142 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:47.394888 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:47.395122 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:47.395198 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:47.397922 (+ 2724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:57739: BlockingRecv error: recv got EOF from 127.1.125.194:57739 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 07:20:47 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:47 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:47.616914 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:47.611138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:54647)
0810 07:20:47.611464 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:47.611477 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:47.611635 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:47.611916 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:47.611919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:47.612142 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:47.612241 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:47.612248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:47.613094 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:47.613099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:47.613674 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:47.613683 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:47.614179 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:47.614182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:47.614257 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:47.614737 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:47.614749 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:47.616780 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":82,"threads_started":1}
W20260810 07:20:47.616981 15344 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:47.611215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:54647 (local address 127.1.125.254:40525)
0810 07:20:47.611376 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:47.611381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:47.611557 (+ 176us) server_negotiation.cc:407] Connection header received
0810 07:20:47.611740 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:47.611745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:47.611799 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:47.611887 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:47.612367 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:47.612979 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:47.613801 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:47.614033 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:47.614152 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:47.616886 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:54647: BlockingRecv error: recv got EOF from 127.1.125.195:54647 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 10 07:20:48 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:48 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:48.210940 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:48.205398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:37267)
0810 07:20:48.205742 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:48.205762 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:48.205877 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:48.206143 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:48.206145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:48.206339 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:48.206441 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:48.206447 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:48.207201 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:48.207203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:48.207749 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:48.207755 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:48.208266 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:48.208269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:48.208333 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:48.208893 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:48.208905 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:48.210812 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":85,"threads_started":1}
W20260810 07:20:48.211095 15349 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:48.205613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:37267 (local address 127.1.125.254:40525)
0810 07:20:48.205866 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:48.205871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:48.205887 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:20:48.205949 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:48.205951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:48.205996 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:48.206088 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:48.206576 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:48.207103 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:48.207890 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:48.208125 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:48.208248 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:48.210947 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:37267: BlockingRecv error: recv got EOF from 127.1.125.193:37267 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":86,"threads_started":1}
Aug 10 07:20:48 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:48 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:48.404430 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:48.398835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:36837)
0810 07:20:48.399171 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:48.399188 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:48.399332 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:48.399618 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:48.399621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:48.399818 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:48.399920 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:48.399926 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:48.400730 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:48.400732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:48.401317 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:48.401326 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:48.401757 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:48.401759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:48.401822 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:48.402358 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:48.402370 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:48.404302 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":90,"threads_started":1}
W20260810 07:20:48.404726 15349 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:48.399017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:36837 (local address 127.1.125.254:40525)
0810 07:20:48.399164 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:48.399169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:48.399309 (+ 140us) server_negotiation.cc:407] Connection header received
0810 07:20:48.399452 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:48.399454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:48.399511 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:48.399577 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:48.400026 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:48.400633 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:48.401422 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:48.401658 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:48.401738 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:48.404446 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:36837: BlockingRecv error: recv got EOF from 127.1.125.194:36837 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 07:20:48 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:48 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:48.623816 15356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:48.617928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:52343)
0810 07:20:48.618212 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:48.618225 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:48.618365 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:48.618647 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:48.618650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:48.618873 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:48.618976 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:48.618983 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:48.619763 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:48.619767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:48.620428 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:48.620438 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:48.620982 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:48.620985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:48.621079 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:48.621692 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:48.621705 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:48.623629 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":88,"threads_started":1}
W20260810 07:20:48.623864 15349 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:48.617927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:52343 (local address 127.1.125.254:40525)
0810 07:20:48.618107 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:48.618113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:48.618304 (+ 191us) server_negotiation.cc:407] Connection header received
0810 07:20:48.618466 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:48.618469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:48.618528 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:48.618615 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:48.619106 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:48.619650 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:48.620614 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:48.620856 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:48.620967 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:48.623760 (+ 2793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:52343: BlockingRecv error: recv got EOF from 127.1.125.195:52343 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 07:20:49 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:49 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:49.222029 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:49.211870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:45493)
0810 07:20:49.212187 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:49.212203 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:49.212299 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:49.212568 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:49.212571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:49.212770 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:49.212884 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:49.212891 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:49.213627 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:49.213629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:49.214248 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:49.214257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:49.214701 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:49.214704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:49.214766 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:49.219829 (+ 5063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:49.219842 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:49.221886 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":95,"threads_started":1}
W20260810 07:20:49.222186 15363 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:49.211995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:45493 (local address 127.1.125.254:40525)
0810 07:20:49.212340 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:49.212344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:49.212357 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:20:49.212394 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:49.212396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:49.212439 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:49.212546 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:49.212991 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:49.213532 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:49.214347 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:49.214606 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:49.214686 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:49.222049 (+ 7363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:45493: BlockingRecv error: recv got EOF from 127.1.125.193:45493 (error 108)
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":95,"threads_started":1}
Aug 10 07:20:49 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:49 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:49.411350 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:49.405428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:46469)
0810 07:20:49.405764 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:49.405784 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:49.405904 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:49.406226 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:49.406229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:49.406448 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:49.406547 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:49.406553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:49.407371 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:49.407377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:49.408122 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:49.408132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:49.408695 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:49.408698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:49.408788 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:49.409274 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:49.409285 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:49.411215 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":79,"threads_started":1}
W20260810 07:20:49.411522 15363 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:49.405695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:46469 (local address 127.1.125.254:40525)
0810 07:20:49.405864 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:49.405869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:49.405884 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:20:49.406053 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:49.406056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:49.406109 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:49.406201 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:49.406669 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:49.407257 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:49.408239 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:49.408574 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:49.408689 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:49.411357 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:46469: BlockingRecv error: recv got EOF from 127.1.125.194:46469 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 07:20:49 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:49 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.195@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:49.631246 15365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:49.624763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.195:38013)
0810 07:20:49.625168 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:49.625184 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:49.625347 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:49.625670 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:49.625673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:49.625921 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:49.626037 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:49.626044 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:49.626954 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:49.626957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:49.627534 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:49.627542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:49.628054 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:49.628058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:49.628152 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:49.628708 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:49.628721 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:49.631063 (+ 2342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":99,"threads_started":1}
W20260810 07:20:49.631322 15363 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:49.624906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:38013 (local address 127.1.125.254:40525)
0810 07:20:49.625105 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:49.625111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:49.625274 (+ 163us) server_negotiation.cc:407] Connection header received
0810 07:20:49.625468 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:49.625473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:49.625536 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:49.625638 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:49.626179 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:49.626765 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:49.627671 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:49.627922 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:49.628047 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:49.631209 (+ 3162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.195:38013: BlockingRecv error: recv got EOF from 127.1.125.195:38013 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 10 07:20:50 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:50 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.193@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:50.228909 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:50.222983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.193:56097)
0810 07:20:50.223329 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:50.223345 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:50.223438 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:50.223759 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:50.223762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:50.223961 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:50.224068 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:50.224074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:50.224884 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:50.224886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:50.225464 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:50.225472 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:50.225918 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:50.225921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:50.226008 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:50.226592 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:50.226604 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:50.228713 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":99,"threads_started":1}
W20260810 07:20:50.228976 15367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:50.223150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:56097 (local address 127.1.125.254:40525)
0810 07:20:50.223404 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:50.223408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:50.223423 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:20:50.223542 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:50.223545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:50.223599 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:50.223726 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:50.224177 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:50.224787 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:50.225589 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:50.225818 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:50.225901 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:50.228845 (+ 2944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.193:56097: BlockingRecv error: recv got EOF from 127.1.125.193:56097 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":68,"threads_started":1}
Aug 10 07:20:50 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:20:50 dist-test-slave-44d2 krb5kdc[13813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.125.194@KRBTEST.COM for oryx/127.1.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:20:50.418383 15368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:50.412305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40525 (local address 127.1.125.194:36519)
0810 07:20:50.412625 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:50.412639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:50.412740 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:50.412977 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:50.412979 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:50.413179 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:50.413282 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:50.413287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:50.414087 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:50.414090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:50.414749 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:50.414758 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:50.415247 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:50.415250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:50.415348 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:50.415937 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:50.415961 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:50.418250 (+ 2289us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.125.254:40525: Server oryx/127.1.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":74,"threads_started":1}
W20260810 07:20:50.418557 15367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:50.412429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:36519 (local address 127.1.125.254:40525)
0810 07:20:50.412618 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:50.412622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:50.412709 (+ 87us) server_negotiation.cc:407] Connection header received
0810 07:20:50.412821 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:50.412824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:50.412868 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:50.412937 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:50.413384 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:50.413981 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:50.414870 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:50.415111 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:50.415207 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:50.418418 (+ 3211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.125.194:36519: BlockingRecv error: recv got EOF from 127.1.125.194:36519 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 07:20:50.522807 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 14660
I20260810 07:20:50.541642 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 14887
I20260810 07:20:50.560464 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 15088
I20260810 07:20:50.577430 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 14559
2026-08-10T07:20:50Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18447 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T07:20:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:50Z Disabled control of system clock
I20260810 07:20:50.614791 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:45863
--webserver_interface=127.1.125.254
--webserver_port=0
--builtin_ntp_servers=127.1.125.212:34585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:45863
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 07:20:50.755719 15376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:50.756029 15376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:50.756068 15376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:50.760171 15376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:50.760236 15376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:50.760254 15376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:50.760273 15376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:50.760352 15376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:50.765182 15376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:34585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:45863
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:45863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:50.766299 15376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:50.767760 15376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:50.776546 15383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:50.776618 15387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:50.776597 15382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:50.776664 15376 server_base.cc:1034] running on GCE node
I20260810 07:20:50.777315 15376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:50.778455 15376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:50.779628 15376 hybrid_clock.cc:648] HybridClock initialized: now 1786346450779598 us; error 43 us; skew 500 ppm
I20260810 07:20:50.781651 15376 webserver.cc:460] Webserver started at http://127.1.125.254:42981/ using document root <none> and password file <none>
I20260810 07:20:50.782238 15376 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:50.782298 15376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:50.782492 15376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:50.784762 15376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "643b1b4d06584da896d25c5a5ac8a7f7"
format_stamp: "Formatted at 2026-08-10 07:20:50 on dist-test-slave-44d2"
server_key: "d0b1771ca3180a65ede0ee3422b9b5d4"
server_key_iv: "20b899f082e57bfffc62203139e51678"
server_key_version: "encryptionkey@0"
I20260810 07:20:50.785301 15376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "643b1b4d06584da896d25c5a5ac8a7f7"
format_stamp: "Formatted at 2026-08-10 07:20:50 on dist-test-slave-44d2"
server_key: "d0b1771ca3180a65ede0ee3422b9b5d4"
server_key_iv: "20b899f082e57bfffc62203139e51678"
server_key_version: "encryptionkey@0"
I20260810 07:20:50.789151 15376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 07:20:50.791654 15395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:50.793007 15376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:50.793156 15376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "643b1b4d06584da896d25c5a5ac8a7f7"
format_stamp: "Formatted at 2026-08-10 07:20:50 on dist-test-slave-44d2"
server_key: "d0b1771ca3180a65ede0ee3422b9b5d4"
server_key_iv: "20b899f082e57bfffc62203139e51678"
server_key_version: "encryptionkey@0"
I20260810 07:20:50.793277 15376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:50.809885 15376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:50.810591 15376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:50.810796 15376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:50.818306 15479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:45863 every 8 connection(s)
I20260810 07:20:50.818305 15376 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:45863
I20260810 07:20:50.819356 15376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:50.821661 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 15376
I20260810 07:20:50.821797 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:50.822150 1527 external_mini_cluster.cc:1428] Setting key fa9b5d368932204fc7cac41e08939ffe
I20260810 07:20:50.822858 15484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:50.829456 15484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7: Bootstrap starting.
I20260810 07:20:50.832021 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:50.823755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46736 (local address 127.1.125.254:45863)
0810 07:20:50.824450 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:50.824466 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:50.824561 (+ 95us) server_negotiation.cc:407] Connection header received
0810 07:20:50.825271 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:50.825289 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:50.825731 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:50.826083 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:50.826532 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:50.827522 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:50.828424 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:50.828977 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:50.829165 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:50.830079 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:50.830086 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:50.830100 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:50.830491 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:50.830546 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:50.830567 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:50.830671 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:50.830856 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 07:20:50.831183 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":145,"threads_started":1}
I20260810 07:20:50.832180 15484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:50.833220 15484 log.cc:824] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:50.835268 15484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7: No bootstrap required, opened a new log
I20260810 07:20:50.838702 15484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643b1b4d06584da896d25c5a5ac8a7f7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45863 } }
I20260810 07:20:50.838954 15484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:50.839018 15484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643b1b4d06584da896d25c5a5ac8a7f7, State: Initialized, Role: FOLLOWER
I20260810 07:20:50.839475 15484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643b1b4d06584da896d25c5a5ac8a7f7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45863 } }
I20260810 07:20:50.839612 15484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:50.839692 15484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:50.839795 15484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:50.840790 15484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643b1b4d06584da896d25c5a5ac8a7f7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45863 } }
I20260810 07:20:50.841132 15484 leader_election.cc:302] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 643b1b4d06584da896d25c5a5ac8a7f7; no voters:
I20260810 07:20:50.841442 15484 leader_election.cc:288] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:50.841570 15491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:50.841827 15491 raft_consensus.cc:686] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [term 1 LEADER]: Becoming Leader. State: Replica: 643b1b4d06584da896d25c5a5ac8a7f7, State: Running, Role: LEADER
I20260810 07:20:50.842223 15491 consensus_queue.cc:237] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643b1b4d06584da896d25c5a5ac8a7f7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45863 } }
I20260810 07:20:50.842751 15484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:50.844120 15492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "643b1b4d06584da896d25c5a5ac8a7f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643b1b4d06584da896d25c5a5ac8a7f7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45863 } } }
I20260810 07:20:50.844295 15492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:50.844120 15493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 643b1b4d06584da896d25c5a5ac8a7f7. Latest consensus state: current_term: 1 leader_uuid: "643b1b4d06584da896d25c5a5ac8a7f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643b1b4d06584da896d25c5a5ac8a7f7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45863 } } }
I20260810 07:20:50.844363 15493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:50.844710 15506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:50.847647 15506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:50.853101 15506 catalog_manager.cc:1347] Generated new cluster ID: 9dd357553bf04ec48085c0bd30e92ae2
I20260810 07:20:50.853178 15506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:50.888780 15506 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:50.889755 15506 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:50.895093 15506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 643b1b4d06584da896d25c5a5ac8a7f7: Generated new TSK 0
I20260810 07:20:50.895776 15506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:20:50.903223 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45863
--builtin_ntp_servers=127.1.125.212:34585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 07:20:51.022307 15517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:51.022624 15517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:51.022706 15517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:51.026867 15517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:51.026975 15517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:51.027076 15517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:51.032253 15517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:34585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:51.033540 15517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:51.034996 15517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:51.045089 15527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:51.045048 15528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:51.045143 15530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:51.045645 15517 server_base.cc:1034] running on GCE node
I20260810 07:20:51.046062 15517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:51.046744 15517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:51.047943 15517 hybrid_clock.cc:648] HybridClock initialized: now 1786346451047894 us; error 52 us; skew 500 ppm
I20260810 07:20:51.050248 15517 webserver.cc:460] Webserver started at http://127.1.125.193:34365/ using document root <none> and password file <none>
I20260810 07:20:51.050925 15517 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:51.050997 15517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:51.051278 15517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:51.053762 15517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "c4f92517d460465eb52781cc5ea72073"
format_stamp: "Formatted at 2026-08-10 07:20:51 on dist-test-slave-44d2"
server_key: "3b257af333dcbb0bab723f51d8314e9c"
server_key_iv: "7e71dfdad7f6f884e0e3f50c6e2d4d0e"
server_key_version: "encryptionkey@0"
I20260810 07:20:51.054415 15517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "c4f92517d460465eb52781cc5ea72073"
format_stamp: "Formatted at 2026-08-10 07:20:51 on dist-test-slave-44d2"
server_key: "3b257af333dcbb0bab723f51d8314e9c"
server_key_iv: "7e71dfdad7f6f884e0e3f50c6e2d4d0e"
server_key_version: "encryptionkey@0"
I20260810 07:20:51.058564 15517 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:51.061230 15541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:51.062547 15517 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:20:51.062716 15517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "c4f92517d460465eb52781cc5ea72073"
format_stamp: "Formatted at 2026-08-10 07:20:51 on dist-test-slave-44d2"
server_key: "3b257af333dcbb0bab723f51d8314e9c"
server_key_iv: "7e71dfdad7f6f884e0e3f50c6e2d4d0e"
server_key_version: "encryptionkey@0"
I20260810 07:20:51.062844 15517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:51.084333 15517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:51.085135 15517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:51.085358 15517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:51.085991 15517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:51.087011 15517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:51.087087 15517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:51.087152 15517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:51.087201 15517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:51.096736 15517 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:36769
I20260810 07:20:51.096778 15701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:36769 every 8 connection(s)
I20260810 07:20:51.097735 15517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:51.101662 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 15517
I20260810 07:20:51.101806 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:51.102129 1527 external_mini_cluster.cc:1428] Setting key 110f50d919f691218158157bf21b64b6
I20260810 07:20:51.104220 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45863
--builtin_ntp_servers=127.1.125.212:34585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 07:20:51.107805 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:51.099682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:50517 (local address 127.1.125.254:45863)
0810 07:20:51.099832 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:51.099836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:51.100998 (+ 1162us) server_negotiation.cc:407] Connection header received
0810 07:20:51.102600 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:51.102603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:51.102673 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:51.102757 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:51.103724 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:51.104287 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:51.105676 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:51.105904 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:51.106023 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:51.107271 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:51.107275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:51.107277 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:51.107290 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:51.107310 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:51.107318 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:51.107398 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:51.107589 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 07:20:51.107660 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 07:20:51.108989 15707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:51.100105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45863 (local address 127.1.125.193:50517)
0810 07:20:51.100863 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:51.100906 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:51.102359 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:51.102945 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:51.102953 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:51.103049 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:51.103573 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:51.103588 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:51.104553 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:51.104559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:51.105532 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:51.105543 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:51.106077 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:51.106081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:51.106270 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:51.106946 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:51.106947 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:51.106993 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:51.107099 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:51.107400 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:51.107413 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:51.107422 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 07:20:51.107566 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 07:20:51.107870 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":116,"threads_started":1}
I20260810 07:20:51.110423 15703 heartbeater.cc:347] Connected to a master server at 127.1.125.254:45863
I20260810 07:20:51.110692 15703 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:51.111245 15703 heartbeater.cc:511] Master 127.1.125.254:45863 requested a full tablet report, sending...
I20260810 07:20:51.112943 15422 ts_manager.cc:194] Registered new tserver with Master: c4f92517d460465eb52781cc5ea72073 (127.1.125.193:36769)
I20260810 07:20:51.114107 15422 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.125.193:50517
W20260810 07:20:51.222702 15708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:51.223050 15708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:51.223125 15708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:51.227312 15708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:51.227425 15708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:51.227536 15708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:51.232496 15708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:34585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:51.233824 15708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:51.235432 15708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:51.246184 15716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:51.246186 15713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:51.246213 15714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:51.246255 15708 server_base.cc:1034] running on GCE node
I20260810 07:20:51.247046 15708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:51.247675 15708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:51.248867 15708 hybrid_clock.cc:648] HybridClock initialized: now 1786346451248853 us; error 33 us; skew 500 ppm
I20260810 07:20:51.250747 15708 webserver.cc:460] Webserver started at http://127.1.125.194:32991/ using document root <none> and password file <none>
I20260810 07:20:51.251350 15708 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:51.251447 15708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:51.251696 15708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:51.253830 15708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "910181e2cb3b460980c8cd24928fa877"
format_stamp: "Formatted at 2026-08-10 07:20:51 on dist-test-slave-44d2"
server_key: "3739a5d972945eca7ceae75c05d8434b"
server_key_iv: "de755549d7815187eaf866782754cd84"
server_key_version: "encryptionkey@0"
I20260810 07:20:51.254415 15708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "910181e2cb3b460980c8cd24928fa877"
format_stamp: "Formatted at 2026-08-10 07:20:51 on dist-test-slave-44d2"
server_key: "3739a5d972945eca7ceae75c05d8434b"
server_key_iv: "de755549d7815187eaf866782754cd84"
server_key_version: "encryptionkey@0"
I20260810 07:20:51.258332 15708 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:51.260946 15724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:51.262295 15708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:51.262446 15708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "910181e2cb3b460980c8cd24928fa877"
format_stamp: "Formatted at 2026-08-10 07:20:51 on dist-test-slave-44d2"
server_key: "3739a5d972945eca7ceae75c05d8434b"
server_key_iv: "de755549d7815187eaf866782754cd84"
server_key_version: "encryptionkey@0"
I20260810 07:20:51.262576 15708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:51.290678 15708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:51.291498 15708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:51.291740 15708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:51.292371 15708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:51.293501 15708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:51.293560 15708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:51.293632 15708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:51.293669 15708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:51.302882 15708 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:37359
I20260810 07:20:51.302959 15901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:37359 every 8 connection(s)
I20260810 07:20:51.304008 15708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:51.313464 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 15708
I20260810 07:20:51.313598 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:51.313594 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:51.306064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:46769 (local address 127.1.125.254:45863)
0810 07:20:51.306235 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:51.306239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:51.307361 (+ 1122us) server_negotiation.cc:407] Connection header received
0810 07:20:51.308843 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:51.308845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:51.308896 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:51.308971 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:51.309903 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:51.310417 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:51.311524 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:51.311773 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:51.311850 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:51.313043 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:51.313046 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:51.313049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:51.313062 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:51.313082 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:51.313092 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:51.313153 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:51.313373 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 07:20:51.313431 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 07:20:51.313966 1527 external_mini_cluster.cc:1428] Setting key 1d138ff358be74e056c0cd762ff26961
I20260810 07:20:51.314733 15904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:51.306518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45863 (local address 127.1.125.194:46769)
0810 07:20:51.307214 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:51.307256 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:51.308642 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:51.309149 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:51.309156 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:51.309240 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:51.309771 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:51.309785 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:51.310538 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:51.310541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:51.311410 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:51.311420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:51.311909 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:51.311912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:51.312054 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:51.312748 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:51.312750 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:51.312789 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:51.312878 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:51.313198 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:51.313210 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:51.313218 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:20:51.313369 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 07:20:51.313717 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":124,"threads_started":1}
I20260810 07:20:51.316004 15902 heartbeater.cc:347] Connected to a master server at 127.1.125.254:45863
I20260810 07:20:51.316254 15902 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:51.316298 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45863
--builtin_ntp_servers=127.1.125.212:34585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 07:20:51.316891 15902 heartbeater.cc:511] Master 127.1.125.254:45863 requested a full tablet report, sending...
I20260810 07:20:51.318027 15422 ts_manager.cc:194] Registered new tserver with Master: 910181e2cb3b460980c8cd24928fa877 (127.1.125.194:37359)
I20260810 07:20:51.318714 15422 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.125.194:46769
W20260810 07:20:51.438998 15907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:51.439275 15907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:51.439347 15907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:51.443688 15907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:51.443745 15907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:51.443854 15907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:51.448642 15907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:34585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:45863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:51.449702 15907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:51.451234 15907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:51.461232 15914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:51.461329 15907 server_base.cc:1034] running on GCE node
W20260810 07:20:51.461351 15917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:51.461447 15915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:51.462014 15907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:51.462687 15907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:51.463841 15907 hybrid_clock.cc:648] HybridClock initialized: now 1786346451463816 us; error 40 us; skew 500 ppm
I20260810 07:20:51.466228 15907 webserver.cc:460] Webserver started at http://127.1.125.195:42633/ using document root <none> and password file <none>
I20260810 07:20:51.466984 15907 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:51.467094 15907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:51.467330 15907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:51.469705 15907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "dc0cadb01afd4acfac9f5e9918f6159f"
format_stamp: "Formatted at 2026-08-10 07:20:51 on dist-test-slave-44d2"
server_key: "a0f731bfef761f4a3d613ae4a7402112"
server_key_iv: "bfffad7c1daa848f9cd8cf14b550e3aa"
server_key_version: "encryptionkey@0"
I20260810 07:20:51.470293 15907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "dc0cadb01afd4acfac9f5e9918f6159f"
format_stamp: "Formatted at 2026-08-10 07:20:51 on dist-test-slave-44d2"
server_key: "a0f731bfef761f4a3d613ae4a7402112"
server_key_iv: "bfffad7c1daa848f9cd8cf14b550e3aa"
server_key_version: "encryptionkey@0"
I20260810 07:20:51.474296 15907 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:51.477020 15924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:51.478296 15907 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:51.478421 15907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "dc0cadb01afd4acfac9f5e9918f6159f"
format_stamp: "Formatted at 2026-08-10 07:20:51 on dist-test-slave-44d2"
server_key: "a0f731bfef761f4a3d613ae4a7402112"
server_key_iv: "bfffad7c1daa848f9cd8cf14b550e3aa"
server_key_version: "encryptionkey@0"
I20260810 07:20:51.478542 15907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:51.497305 15907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:51.498051 15907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:51.498247 15907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:51.498878 15907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:51.499874 15907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:51.499933 15907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:51.500005 15907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:51.500047 15907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:51.509519 15907 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:44279
I20260810 07:20:51.509543 16093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:44279 every 8 connection(s)
I20260810 07:20:51.510490 15907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:51.514653 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 15907
I20260810 07:20:51.514746 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:51.515013 1527 external_mini_cluster.cc:1428] Setting key 8add1b95c55c3560174b10ce8d6a0b38
I20260810 07:20:51.520220 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:51.512454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:60871 (local address 127.1.125.254:45863)
0810 07:20:51.512631 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:51.512635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:51.513723 (+ 1088us) server_negotiation.cc:407] Connection header received
0810 07:20:51.515312 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:51.515316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:51.515382 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:51.515466 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:51.516398 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:51.517023 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:51.518195 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:51.518434 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:51.518549 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:51.519706 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:51.519710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:51.519713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:51.519726 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:51.519749 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:51.519766 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:51.519846 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:51.520015 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 07:20:51.520081 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 07:20:51.521238 16097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:51.512878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45863 (local address 127.1.125.195:60871)
0810 07:20:51.513555 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:51.513599 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:51.515037 (+ 1438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:51.515658 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:51.515665 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:51.515750 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:51.516246 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:51.516260 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:51.517178 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:51.517182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:51.518051 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:51.518061 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:51.518594 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:51.518599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:51.518772 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:51.519439 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:51.519440 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:51.519481 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:51.519545 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:51.519849 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:51.519861 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:51.519871 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 07:20:51.520011 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 07:20:51.520309 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":113,"threads_started":1}
I20260810 07:20:51.522343 16095 heartbeater.cc:347] Connected to a master server at 127.1.125.254:45863
I20260810 07:20:51.522588 16095 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:51.523118 16095 heartbeater.cc:511] Master 127.1.125.254:45863 requested a full tablet report, sending...
I20260810 07:20:51.524143 15422 ts_manager.cc:194] Registered new tserver with Master: dc0cadb01afd4acfac9f5e9918f6159f (127.1.125.195:44279)
I20260810 07:20:51.524736 15422 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.125.195:60871
I20260810 07:20:51.528721 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 07:20:51.531728 15488 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:51.530680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46748 (local address 127.1.125.254:45863)
0810 07:20:51.530828 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:51.530834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:51.531091 (+ 257us) server_negotiation.cc:407] Connection header received
0810 07:20:51.531175 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:51.531178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:51.531218 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:51.531284 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:51.531573 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46748: BlockingRecv error: recv got EOF from 127.0.0.1:46748 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:20:51.531842 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 15517
I20260810 07:20:51.548666 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 15708
I20260810 07:20:51.564174 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 15907
I20260810 07:20:51.581975 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 15376
2026-08-10T07:20:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1004 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T07:20:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:51Z Disabled control of system clock
I20260810 07:20:51.618854 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:33177
--webserver_interface=127.1.125.254
--webserver_port=0
--builtin_ntp_servers=127.1.125.212:45763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:33177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 07:20:51.748656 16109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:51.748937 16109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:51.748976 16109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:51.752985 16109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:51.753046 16109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:51.753062 16109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:51.753078 16109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:51.753093 16109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:51.757956 16109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:33177
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:33177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:51.759039 16109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:51.760403 16109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:20:51.769349 16109 server_base.cc:1034] running on GCE node
W20260810 07:20:51.769301 16118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:51.769315 16124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:51.769295 16119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:51.770112 16109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:51.771266 16109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:51.772517 16109 hybrid_clock.cc:648] HybridClock initialized: now 1786346451772465 us; error 30 us; skew 500 ppm
I20260810 07:20:51.774550 16109 webserver.cc:460] Webserver started at http://127.1.125.254:33899/ using document root <none> and password file <none>
I20260810 07:20:51.775215 16109 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:51.775308 16109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:51.775550 16109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:51.777684 16109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "6a97f38148964d3d90827e25cfcc9d79"
format_stamp: "Formatted at 2026-08-10 07:20:51 on dist-test-slave-44d2"
server_key: "1df5f7ba576d8ae598e6a8c92a816bb2"
server_key_iv: "9fa7aa4349144170e8a06eb9c0435135"
server_key_version: "encryptionkey@0"
I20260810 07:20:51.778268 16109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "6a97f38148964d3d90827e25cfcc9d79"
format_stamp: "Formatted at 2026-08-10 07:20:51 on dist-test-slave-44d2"
server_key: "1df5f7ba576d8ae598e6a8c92a816bb2"
server_key_iv: "9fa7aa4349144170e8a06eb9c0435135"
server_key_version: "encryptionkey@0"
I20260810 07:20:51.782127 16109 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:51.784943 16130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:51.786330 16109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:51.786594 16109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "6a97f38148964d3d90827e25cfcc9d79"
format_stamp: "Formatted at 2026-08-10 07:20:51 on dist-test-slave-44d2"
server_key: "1df5f7ba576d8ae598e6a8c92a816bb2"
server_key_iv: "9fa7aa4349144170e8a06eb9c0435135"
server_key_version: "encryptionkey@0"
I20260810 07:20:51.786727 16109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:51.791649 16109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:51.792294 16109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:51.792524 16109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:51.799501 16109 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:33177
I20260810 07:20:51.799522 16202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:33177 every 8 connection(s)
I20260810 07:20:51.800432 16109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:51.803681 16211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:51.805122 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 16109
I20260810 07:20:51.805291 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:51.805656 1527 external_mini_cluster.cc:1428] Setting key 37dfdd907d47a0cfb2cc82e300ab4198
I20260810 07:20:51.810763 16211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79: Bootstrap starting.
I20260810 07:20:51.812131 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:51.807314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58534 (local address 127.1.125.254:33177)
0810 07:20:51.808101 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:51.808121 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:51.808164 (+ 43us) server_negotiation.cc:407] Connection header received
0810 07:20:51.809026 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:51.809047 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:51.809453 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:51.809834 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:51.810743 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:51.810748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:51.810761 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:51.810775 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:51.810839 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:51.810861 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:51.811048 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:51.811135 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 07:20:51.811402 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":147,"threads_started":1}
I20260810 07:20:51.813380 16211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:51.814132 16211 log.cc:824] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:51.816200 16211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79: No bootstrap required, opened a new log
I20260810 07:20:51.819952 16211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a97f38148964d3d90827e25cfcc9d79" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 33177 } }
I20260810 07:20:51.820211 16211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:51.820307 16211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a97f38148964d3d90827e25cfcc9d79, State: Initialized, Role: FOLLOWER
I20260810 07:20:51.820863 16211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a97f38148964d3d90827e25cfcc9d79" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 33177 } }
I20260810 07:20:51.821022 16211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:51.821096 16211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:51.821197 16211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:51.822139 16211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a97f38148964d3d90827e25cfcc9d79" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 33177 } }
I20260810 07:20:51.822500 16211 leader_election.cc:302] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a97f38148964d3d90827e25cfcc9d79; no voters:
I20260810 07:20:51.822836 16211 leader_election.cc:288] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:51.823073 16217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:51.823345 16217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [term 1 LEADER]: Becoming Leader. State: Replica: 6a97f38148964d3d90827e25cfcc9d79, State: Running, Role: LEADER
I20260810 07:20:51.823648 16217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a97f38148964d3d90827e25cfcc9d79" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 33177 } }
I20260810 07:20:51.824301 16211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:51.825476 16219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a97f38148964d3d90827e25cfcc9d79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a97f38148964d3d90827e25cfcc9d79" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 33177 } } }
I20260810 07:20:51.825614 16219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:51.826015 16220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a97f38148964d3d90827e25cfcc9d79. Latest consensus state: current_term: 1 leader_uuid: "6a97f38148964d3d90827e25cfcc9d79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a97f38148964d3d90827e25cfcc9d79" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 33177 } } }
I20260810 07:20:51.826202 16220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79 [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:51.827019 16234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:51.830065 16234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:51.835995 16234 catalog_manager.cc:1347] Generated new cluster ID: 71f0a06166d9485c8367fed5c6b60b2c
I20260810 07:20:51.836061 16234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:51.855734 16234 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:51.856089 16234 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:51.864223 16234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a97f38148964d3d90827e25cfcc9d79: Generated new TSK 0
I20260810 07:20:51.865034 16234 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:20:51.869211 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:33177
--builtin_ntp_servers=127.1.125.212:45763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 07:20:51.987125 16253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:51.987462 16253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:51.987542 16253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:51.991869 16253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:51.991959 16253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:51.992049 16253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:51.997100 16253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:33177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:51.998286 16253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:51.999743 16253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:52.009928 16259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:52.009963 16262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:52.010078 16253 server_base.cc:1034] running on GCE node
W20260810 07:20:52.009994 16260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:52.010783 16253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:52.011586 16253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:52.012804 16253 hybrid_clock.cc:648] HybridClock initialized: now 1786346452012756 us; error 57 us; skew 500 ppm
I20260810 07:20:52.014854 16253 webserver.cc:460] Webserver started at http://127.1.125.193:39993/ using document root <none> and password file <none>
I20260810 07:20:52.015501 16253 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:52.015600 16253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:52.015841 16253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:52.018016 16253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "5f199c74734c464daee89ae3c2dbc537"
format_stamp: "Formatted at 2026-08-10 07:20:52 on dist-test-slave-44d2"
server_key: "591b28e0d01fb610152eb33baaf52f44"
server_key_iv: "e79fa93f4be5e6d2f0de99ebd271917c"
server_key_version: "encryptionkey@0"
I20260810 07:20:52.018617 16253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "5f199c74734c464daee89ae3c2dbc537"
format_stamp: "Formatted at 2026-08-10 07:20:52 on dist-test-slave-44d2"
server_key: "591b28e0d01fb610152eb33baaf52f44"
server_key_iv: "e79fa93f4be5e6d2f0de99ebd271917c"
server_key_version: "encryptionkey@0"
I20260810 07:20:52.022454 16253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:52.025478 16271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:52.026957 16253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:52.027099 16253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "5f199c74734c464daee89ae3c2dbc537"
format_stamp: "Formatted at 2026-08-10 07:20:52 on dist-test-slave-44d2"
server_key: "591b28e0d01fb610152eb33baaf52f44"
server_key_iv: "e79fa93f4be5e6d2f0de99ebd271917c"
server_key_version: "encryptionkey@0"
I20260810 07:20:52.027223 16253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:52.032023 16253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:52.032794 16253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:52.032986 16253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:52.033550 16253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:52.034502 16253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:52.034574 16253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:52.034648 16253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:52.034699 16253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:52.043908 16253 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:39889
I20260810 07:20:52.043943 16444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:39889 every 8 connection(s)
I20260810 07:20:52.045061 16253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:52.047227 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 16253
I20260810 07:20:52.047329 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:52.047672 1527 external_mini_cluster.cc:1428] Setting key 733102cafa359c3a3f04991180df056e
I20260810 07:20:52.049847 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:33177
--builtin_ntp_servers=127.1.125.212:45763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 07:20:52.051846 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:52.046970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:54527 (local address 127.1.125.254:33177)
0810 07:20:52.047160 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:52.047166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:52.048371 (+ 1205us) server_negotiation.cc:407] Connection header received
0810 07:20:52.049887 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:52.049891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:52.049938 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:52.050036 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:52.051249 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:52.051255 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:52.051262 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:52.051279 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:52.051306 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:52.051319 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:52.051419 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:52.051607 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 07:20:52.051672 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 07:20:52.053159 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:52.047445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:33177 (local address 127.1.125.193:54527)
0810 07:20:52.048207 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:52.048259 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:52.049660 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:52.050268 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:52.050275 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:52.050363 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:52.050969 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:52.050971 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:52.051012 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:52.051084 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:52.051411 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:52.051418 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:52.051422 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:20:52.051632 (+ 210us) client_negotiation.cc:240] Negotiation successful
0810 07:20:52.051908 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":136,"threads_started":1}
I20260810 07:20:52.054610 16445 heartbeater.cc:347] Connected to a master server at 127.1.125.254:33177
I20260810 07:20:52.054867 16445 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:52.055358 16445 heartbeater.cc:511] Master 127.1.125.254:33177 requested a full tablet report, sending...
I20260810 07:20:52.057195 16152 ts_manager.cc:194] Registered new tserver with Master: 5f199c74734c464daee89ae3c2dbc537 (127.1.125.193:39889)
W20260810 07:20:52.171108 16448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:52.171371 16448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:52.171418 16448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:52.175366 16448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:52.175431 16448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:52.175485 16448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:52.180184 16448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:33177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:52.181314 16448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:52.182718 16448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:52.193413 16456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:52.193318 16460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:52.193318 16455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:52.193758 16448 server_base.cc:1034] running on GCE node
I20260810 07:20:52.194391 16448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:52.195147 16448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:52.196343 16448 hybrid_clock.cc:648] HybridClock initialized: now 1786346452196322 us; error 27 us; skew 500 ppm
I20260810 07:20:52.198567 16448 webserver.cc:460] Webserver started at http://127.1.125.194:38563/ using document root <none> and password file <none>
I20260810 07:20:52.199224 16448 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:52.199296 16448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:52.199550 16448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:52.201887 16448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "729b5dd3e0204ee1a33e498efe2e0603"
format_stamp: "Formatted at 2026-08-10 07:20:52 on dist-test-slave-44d2"
server_key: "92110e8c5add4db2f2efcfb844756039"
server_key_iv: "19f74c4b61fd742b9fc6296fe702b0f0"
server_key_version: "encryptionkey@0"
I20260810 07:20:52.202515 16448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "729b5dd3e0204ee1a33e498efe2e0603"
format_stamp: "Formatted at 2026-08-10 07:20:52 on dist-test-slave-44d2"
server_key: "92110e8c5add4db2f2efcfb844756039"
server_key_iv: "19f74c4b61fd742b9fc6296fe702b0f0"
server_key_version: "encryptionkey@0"
I20260810 07:20:52.206758 16448 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:52.209751 16471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:52.211050 16448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:52.211181 16448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "729b5dd3e0204ee1a33e498efe2e0603"
format_stamp: "Formatted at 2026-08-10 07:20:52 on dist-test-slave-44d2"
server_key: "92110e8c5add4db2f2efcfb844756039"
server_key_iv: "19f74c4b61fd742b9fc6296fe702b0f0"
server_key_version: "encryptionkey@0"
I20260810 07:20:52.211287 16448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:52.217144 16448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:52.217878 16448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:52.218098 16448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:52.218703 16448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:52.219774 16448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:52.219853 16448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:52.219934 16448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:52.219991 16448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:52.231894 16448 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:44555
I20260810 07:20:52.231914 16643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:44555 every 8 connection(s)
I20260810 07:20:52.233201 16448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:52.238494 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 16448
I20260810 07:20:52.238605 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:52.238956 1527 external_mini_cluster.cc:1428] Setting key b83b24a670f76798d8c5e5926e5f4a13
I20260810 07:20:52.239995 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:52.235296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:36809 (local address 127.1.125.254:33177)
0810 07:20:52.235479 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:52.235484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:52.236754 (+ 1270us) server_negotiation.cc:407] Connection header received
0810 07:20:52.238123 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:52.238125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:52.238165 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:52.238298 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:52.239505 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:52.239509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:52.239512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:52.239525 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:52.239548 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:52.239565 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:52.239671 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:52.239840 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 07:20:52.239882 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 07:20:52.241042 16648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:52.235725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:33177 (local address 127.1.125.194:36809)
0810 07:20:52.236568 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:52.236635 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:52.237922 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:52.238455 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:52.238462 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:52.238575 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:52.239238 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:52.239240 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:52.239281 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:52.239351 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:52.239656 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:52.239665 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:52.239672 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 07:20:52.239873 (+ 201us) client_negotiation.cc:240] Negotiation successful
0810 07:20:52.240129 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":115,"threads_started":1}
I20260810 07:20:52.241096 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:33177
--builtin_ntp_servers=127.1.125.212:45763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 07:20:52.243942 16646 heartbeater.cc:347] Connected to a master server at 127.1.125.254:33177
I20260810 07:20:52.244204 16646 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:52.244721 16646 heartbeater.cc:511] Master 127.1.125.254:33177 requested a full tablet report, sending...
I20260810 07:20:52.245921 16152 ts_manager.cc:194] Registered new tserver with Master: 729b5dd3e0204ee1a33e498efe2e0603 (127.1.125.194:44555)
W20260810 07:20:52.363327 16649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:52.363740 16649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:52.363818 16649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:52.368160 16649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:52.368216 16649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:52.368306 16649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:52.373184 16649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:33177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:52.374325 16649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:52.375840 16649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:52.386193 16655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:52.386219 16657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:52.386173 16659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:52.386394 16649 server_base.cc:1034] running on GCE node
I20260810 07:20:52.387068 16649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:52.387856 16649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:52.389062 16649 hybrid_clock.cc:648] HybridClock initialized: now 1786346452389016 us; error 49 us; skew 500 ppm
I20260810 07:20:52.391290 16649 webserver.cc:460] Webserver started at http://127.1.125.195:36191/ using document root <none> and password file <none>
I20260810 07:20:52.391932 16649 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:52.391999 16649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:52.392174 16649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:52.394402 16649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "2664bfbff226432d9718b13686399d2b"
format_stamp: "Formatted at 2026-08-10 07:20:52 on dist-test-slave-44d2"
server_key: "b083a2e64962915b9a930eaec212957a"
server_key_iv: "c478f108c7ed9f0a0b2f420c4a797bc5"
server_key_version: "encryptionkey@0"
I20260810 07:20:52.394989 16649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "2664bfbff226432d9718b13686399d2b"
format_stamp: "Formatted at 2026-08-10 07:20:52 on dist-test-slave-44d2"
server_key: "b083a2e64962915b9a930eaec212957a"
server_key_iv: "c478f108c7ed9f0a0b2f420c4a797bc5"
server_key_version: "encryptionkey@0"
I20260810 07:20:52.399048 16649 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:52.402072 16668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:52.403492 16649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 07:20:52.403626 16649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "2664bfbff226432d9718b13686399d2b"
format_stamp: "Formatted at 2026-08-10 07:20:52 on dist-test-slave-44d2"
server_key: "b083a2e64962915b9a930eaec212957a"
server_key_iv: "c478f108c7ed9f0a0b2f420c4a797bc5"
server_key_version: "encryptionkey@0"
I20260810 07:20:52.403743 16649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:52.408692 16649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:52.409391 16649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:52.409575 16649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:52.410147 16649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:52.411088 16649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:52.411144 16649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:52.411211 16649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:52.411247 16649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:52.420784 16649 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:44513
I20260810 07:20:52.420810 16848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:44513 every 8 connection(s)
I20260810 07:20:52.421737 16649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:20:52.428653 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:52.423718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:52097 (local address 127.1.125.254:33177)
0810 07:20:52.423897 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:52.423903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:52.425388 (+ 1485us) server_negotiation.cc:407] Connection header received
0810 07:20:52.426780 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:52.426784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:52.426832 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:52.426913 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:52.428110 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:52.428113 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:52.428115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:20:52.428127 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:52.428146 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:20:52.428154 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:52.428236 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:52.428404 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 07:20:52.428450 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 07:20:52.429468 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:52.424283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:33177 (local address 127.1.125.195:52097)
0810 07:20:52.425196 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:52.425271 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:52.426576 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:52.427102 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:52.427109 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:52.427197 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:52.427837 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:20:52.427838 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:52.427881 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:20:52.427952 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:52.428232 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:52.428238 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:52.428242 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:20:52.428373 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 07:20:52.428622 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":192,"threads_started":1}
I20260810 07:20:52.429553 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 16649
I20260810 07:20:52.429701 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:52.430066 1527 external_mini_cluster.cc:1428] Setting key 9aa988cc6348bb71b0b92484e838bf50
I20260810 07:20:52.430923 16849 heartbeater.cc:347] Connected to a master server at 127.1.125.254:33177
I20260810 07:20:52.431191 16849 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:52.431615 16849 heartbeater.cc:511] Master 127.1.125.254:33177 requested a full tablet report, sending...
I20260810 07:20:52.432776 16152 ts_manager.cc:194] Registered new tserver with Master: 2664bfbff226432d9718b13686399d2b (127.1.125.195:44513)
I20260810 07:20:52.447558 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 07:20:52.453714 16215 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:20:52.450314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58538 (local address 127.1.125.254:33177)
0810 07:20:52.450479 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:52.450485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:52.450727 (+ 242us) server_negotiation.cc:407] Connection header received
0810 07:20:52.450904 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:52.450908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:52.450962 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:52.451075 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:52.453559 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58538: BlockingRecv error: recv got EOF from 127.0.0.1:58538 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:20:52.454916 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 16253
I20260810 07:20:52.482131 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 16448
I20260810 07:20:52.495788 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 16649
I20260810 07:20:52.513768 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 16109
2026-08-10T07:20:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (935 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:52 dist-test-slave-44d2 krb5kdc[16862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:52 dist-test-slave-44d2 krb5kdc[16862](info): set up 2 sockets
Aug 10 07:20:52 dist-test-slave-44d2 krb5kdc[16862](info): commencing operation
krb5kdc: starting...
W20260810 07:20:53.773309 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:20:53 dist-test-slave-44d2 krb5kdc[16862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:20:53.908308 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:36789
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:35175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:36789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:54.026350 16884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:54.026651 16884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:54.026692 16884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:54.030782 16884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:20:54.030846 16884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:54.030866 16884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:54.030882 16884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:20:54.030897 16884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:20:54.036324 16884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:35175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:36789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:36789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:54.037498 16884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:54.038575 16884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:54.047541 16892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:54.047554 16895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:54.047551 16890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:54.047828 16884 server_base.cc:1034] running on GCE node
I20260810 07:20:54.048448 16884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:54.049605 16884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:54.050793 16884 hybrid_clock.cc:648] HybridClock initialized: now 1786346454050758 us; error 46 us; skew 500 ppm
Aug 10 07:20:54 dist-test-slave-44d2 krb5kdc[16862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:54.053995 16884 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:20:54.055011 16884 webserver.cc:460] Webserver started at http://127.1.125.254:45871/ using document root <none> and password file <none>
I20260810 07:20:54.055688 16884 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:54.055786 16884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:54.056051 16884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:54.058254 16884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "d9f731c54e734957b10913a8a816424f"
format_stamp: "Formatted at 2026-08-10 07:20:54 on dist-test-slave-44d2"
server_key: "c68d906cc668b380507d94e3f0086e23"
server_key_iv: "ec8ee1c3d4819f180615d8793b649d7a"
server_key_version: "encryptionkey@0"
I20260810 07:20:54.058883 16884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "d9f731c54e734957b10913a8a816424f"
format_stamp: "Formatted at 2026-08-10 07:20:54 on dist-test-slave-44d2"
server_key: "c68d906cc668b380507d94e3f0086e23"
server_key_iv: "ec8ee1c3d4819f180615d8793b649d7a"
server_key_version: "encryptionkey@0"
I20260810 07:20:54.062862 16884 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260810 07:20:54.065565 16904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:54.066956 16884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:20:54.067133 16884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "d9f731c54e734957b10913a8a816424f"
format_stamp: "Formatted at 2026-08-10 07:20:54 on dist-test-slave-44d2"
server_key: "c68d906cc668b380507d94e3f0086e23"
server_key_iv: "ec8ee1c3d4819f180615d8793b649d7a"
server_key_version: "encryptionkey@0"
I20260810 07:20:54.067265 16884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:54.091248 16884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:54.095171 16884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:54.095427 16884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:54.103825 16884 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:36789
I20260810 07:20:54.104251 16981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:36789 every 8 connection(s)
I20260810 07:20:54.105037 16884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:20:54.106602 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 16884
I20260810 07:20:54.106766 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:20:54.107090 1527 external_mini_cluster.cc:1428] Setting key eca7ba46ec4299aa7a57bec9da224409
I20260810 07:20:54.108736 16983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:54.116801 16983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f: Bootstrap starting.
Aug 10 07:20:54 dist-test-slave-44d2 krb5kdc[16862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:54.119807 16983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:54.120888 16983 log.cc:824] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:54.123332 16983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f: No bootstrap required, opened a new log
I20260810 07:20:54.124164 16986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.108847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34338 (local address 127.1.125.254:36789)
0810 07:20:54.109321 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:54.109342 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:54.109390 (+ 48us) server_negotiation.cc:407] Connection header received
0810 07:20:54.110362 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:54.110391 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:54.110849 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:54.111269 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:54.112017 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.113735 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.114633 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.115473 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.115649 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.118747 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:54.118753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:54.118766 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:54.118796 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:54.121061 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:54.121502 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:54.121509 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:54.121515 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:54.121666 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:54.121999 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:54.122002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:54.122004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:54.122433 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:54.122594 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:54.122988 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 07:20:54.123326 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":115,"threads_started":1}
I20260810 07:20:54.127300 16983 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f731c54e734957b10913a8a816424f" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36789 } }
I20260810 07:20:54.127552 16983 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:54.127632 16983 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9f731c54e734957b10913a8a816424f, State: Initialized, Role: FOLLOWER
I20260810 07:20:54.128067 16983 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f731c54e734957b10913a8a816424f" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36789 } }
I20260810 07:20:54.128223 16983 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:20:54.128309 16983 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:20:54.128430 16983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:54.129361 16983 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f731c54e734957b10913a8a816424f" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36789 } }
I20260810 07:20:54.129717 16983 leader_election.cc:302] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9f731c54e734957b10913a8a816424f; no voters:
I20260810 07:20:54.130048 16983 leader_election.cc:288] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:20:54.130190 16989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:54.130455 16989 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [term 1 LEADER]: Becoming Leader. State: Replica: d9f731c54e734957b10913a8a816424f, State: Running, Role: LEADER
I20260810 07:20:54.130784 16989 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f731c54e734957b10913a8a816424f" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36789 } }
I20260810 07:20:54.131410 16983 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:20:54.132607 16990 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9f731c54e734957b10913a8a816424f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f731c54e734957b10913a8a816424f" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36789 } } }
I20260810 07:20:54.132738 16990 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:54.133157 17011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:20:54.133138 16993 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9f731c54e734957b10913a8a816424f. Latest consensus state: current_term: 1 leader_uuid: "d9f731c54e734957b10913a8a816424f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f731c54e734957b10913a8a816424f" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 36789 } } }
I20260810 07:20:54.133493 16993 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f [sys.catalog]: This master's current role is: LEADER
I20260810 07:20:54.135910 17011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:20:54.141893 17011 catalog_manager.cc:1347] Generated new cluster ID: 4a402204502d4b8286981dc8b91ff5b2
I20260810 07:20:54.141966 17011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:20:54.154652 17011 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:20:54.155687 17011 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:20:54.160430 17011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f: Generated new TSK 0
I20260810 07:20:54.161293 17011 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:20:54.195314 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:36789
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:35175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:54.317225 17032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:54.317536 17032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:54.317610 17032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:54.321655 17032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:54.321745 17032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:54.321841 17032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:20:54.327107 17032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:35175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:36789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:54.328228 17032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:54.329317 17032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:54.339495 17039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:54.339574 17042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:54.339636 17032 server_base.cc:1034] running on GCE node
W20260810 07:20:54.339814 17044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:54.340243 17032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:54.340998 17032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:54.342190 17032 hybrid_clock.cc:648] HybridClock initialized: now 1786346454342157 us; error 43 us; skew 500 ppm
Aug 10 07:20:54 dist-test-slave-44d2 krb5kdc[16862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:54.350337 17032 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:20:54.351363 17032 webserver.cc:460] Webserver started at http://127.1.125.193:46569/ using document root <none> and password file <none>
I20260810 07:20:54.351935 17032 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:54.352000 17032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:54.352192 17032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:54.354393 17032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "b4a18d1fa9fe4130978f759354c5557d"
format_stamp: "Formatted at 2026-08-10 07:20:54 on dist-test-slave-44d2"
server_key: "47b53201a8ee4aec104c81e4832114ee"
server_key_iv: "035b50af98ff02fa319bbe62f88b4cd3"
server_key_version: "encryptionkey@0"
I20260810 07:20:54.354923 17032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "b4a18d1fa9fe4130978f759354c5557d"
format_stamp: "Formatted at 2026-08-10 07:20:54 on dist-test-slave-44d2"
server_key: "47b53201a8ee4aec104c81e4832114ee"
server_key_iv: "035b50af98ff02fa319bbe62f88b4cd3"
server_key_version: "encryptionkey@0"
I20260810 07:20:54.358860 17032 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:20:54.361333 17055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:54.362589 17032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:20:54.362725 17032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "b4a18d1fa9fe4130978f759354c5557d"
format_stamp: "Formatted at 2026-08-10 07:20:54 on dist-test-slave-44d2"
server_key: "47b53201a8ee4aec104c81e4832114ee"
server_key_iv: "035b50af98ff02fa319bbe62f88b4cd3"
server_key_version: "encryptionkey@0"
I20260810 07:20:54.362836 17032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:54.380214 17032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:54.383221 17032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:54.383438 17032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:54.384099 17032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:54.385035 17032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:54.385092 17032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:54.385180 17032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:54.385213 17032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:54.394681 17032 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:43955
I20260810 07:20:54.394713 17239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:43955 every 8 connection(s)
I20260810 07:20:54.395641 17032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:20:54.403923 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 17032
I20260810 07:20:54.404035 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:20:54.404400 1527 external_mini_cluster.cc:1428] Setting key 6d9f182b82c460c63a66abcea90b3ec4
Aug 10 07:20:54 dist-test-slave-44d2 krb5kdc[16862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:54.409909 16986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.397626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:53529 (local address 127.1.125.254:36789)
0810 07:20:54.397809 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:54.397815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:54.398728 (+ 913us) server_negotiation.cc:407] Connection header received
0810 07:20:54.400076 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:54.400085 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:54.400135 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:54.400203 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:54.401395 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.401931 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.403031 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.403272 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.403350 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.406258 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:54.406262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:54.406264 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:54.406287 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:54.407751 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:54.408295 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:54.408304 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:54.408305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:54.408383 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:54.408771 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:54.408775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:54.408777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:54.408962 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:54.409073 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:54.409603 (+ 530us) server_negotiation.cc:299] Negotiation successful
0810 07:20:54.409734 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 07:20:54.410693 17242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.398077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:36789 (local address 127.1.125.193:53529)
0810 07:20:54.398560 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:54.398604 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:54.399853 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:54.400371 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:54.400377 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:54.400761 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:54.401266 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:54.401280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.402053 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.402056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:54.402925 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:54.402934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.403404 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.403407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:54.403551 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.404283 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:54.404305 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:54.406071 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:54.407881 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:54.407888 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:54.407903 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:54.408180 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:54.408502 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:54.408508 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:54.408513 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:54.408670 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:54.409073 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:54.409092 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:54.409334 (+ 242us) client_negotiation.cc:769] Sending connection context
0810 07:20:54.409523 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 07:20:54.409849 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":117,"threads_started":1}
I20260810 07:20:54.412088 17240 heartbeater.cc:347] Connected to a master server at 127.1.125.254:36789
I20260810 07:20:54.412333 17240 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:54.412890 17240 heartbeater.cc:511] Master 127.1.125.254:36789 requested a full tablet report, sending...
I20260810 07:20:54.414362 16929 ts_manager.cc:194] Registered new tserver with Master: b4a18d1fa9fe4130978f759354c5557d (127.1.125.193:43955)
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:20:54.415565 16929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:53529
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:20:54.433812 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:36789
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:35175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:54.553803 17247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:54.554140 17247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:54.554219 17247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:54.558321 17247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:54.558426 17247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:54.558529 17247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:20:54.564630 17247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:35175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:36789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:54.565917 17247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:54.567128 17247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:54.577709 17254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:54.577702 17255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:54.577756 17247 server_base.cc:1034] running on GCE node
W20260810 07:20:54.577704 17257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:54.578368 17247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:54.579054 17247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:54.580261 17247 hybrid_clock.cc:648] HybridClock initialized: now 1786346454580233 us; error 46 us; skew 500 ppm
Aug 10 07:20:54 dist-test-slave-44d2 krb5kdc[16862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:54.583230 17247 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:20:54.584170 17247 webserver.cc:460] Webserver started at http://127.1.125.194:44943/ using document root <none> and password file <none>
I20260810 07:20:54.584830 17247 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:54.584936 17247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:54.585201 17247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:54.587378 17247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "9441e032e003499393e5797cc70cb0bf"
format_stamp: "Formatted at 2026-08-10 07:20:54 on dist-test-slave-44d2"
server_key: "78c347897765e56ff2c7361f4698e84c"
server_key_iv: "8a3c0142b2e2ffc79524e7c8ee60e7fd"
server_key_version: "encryptionkey@0"
I20260810 07:20:54.587980 17247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "9441e032e003499393e5797cc70cb0bf"
format_stamp: "Formatted at 2026-08-10 07:20:54 on dist-test-slave-44d2"
server_key: "78c347897765e56ff2c7361f4698e84c"
server_key_iv: "8a3c0142b2e2ffc79524e7c8ee60e7fd"
server_key_version: "encryptionkey@0"
I20260810 07:20:54.591881 17247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260810 07:20:54.594565 17268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:54.595791 17247 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:20:54.595947 17247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "9441e032e003499393e5797cc70cb0bf"
format_stamp: "Formatted at 2026-08-10 07:20:54 on dist-test-slave-44d2"
server_key: "78c347897765e56ff2c7361f4698e84c"
server_key_iv: "8a3c0142b2e2ffc79524e7c8ee60e7fd"
server_key_version: "encryptionkey@0"
I20260810 07:20:54.596076 17247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:54.611024 17247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:54.614665 17247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:54.614880 17247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:54.615531 17247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:54.616415 17247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:54.616490 17247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:54.616632 17247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:54.616689 17247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:54.626932 17247 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:37551
I20260810 07:20:54.626972 17445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:37551 every 8 connection(s)
I20260810 07:20:54.627903 17247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:20:54.631664 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 17247
I20260810 07:20:54.631790 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:20:54.632130 1527 external_mini_cluster.cc:1428] Setting key 52e96da35d4fcf45d8ed1c356cb2c266
Aug 10 07:20:54 dist-test-slave-44d2 krb5kdc[16862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
I20260810 07:20:54.644158 16986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.629835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:57655 (local address 127.1.125.254:36789)
0810 07:20:54.629994 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:54.629998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:54.630856 (+ 858us) server_negotiation.cc:407] Connection header received
0810 07:20:54.632764 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:54.632767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:54.632832 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:54.632911 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:54.634327 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.634850 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.636534 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.636779 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.636900 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.639952 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:54.639958 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:54.639963 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:54.639999 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:54.641847 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:54.642503 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:54.642508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:54.642513 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:54.642620 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:54.642966 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:54.642970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:54.642972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:54.643232 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:54.643351 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:54.643846 (+ 495us) server_negotiation.cc:299] Negotiation successful
0810 07:20:54.643968 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:20:54.645222 17448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.630218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:36789 (local address 127.1.125.194:57655)
0810 07:20:54.630697 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:54.630743 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:54.632440 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:54.633101 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:54.633108 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:54.633477 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:54.634161 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:54.634187 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.634988 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.634994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:54.636359 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:54.636370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.636945 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.636949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:54.637133 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.637781 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:54.637799 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:54.639761 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:54.642022 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:54.642031 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:54.642047 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:54.642388 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:54.642728 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:54.642730 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:54.642732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:54.642865 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:54.643344 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:54.643349 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:54.643584 (+ 235us) client_negotiation.cc:769] Sending connection context
0810 07:20:54.643793 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 07:20:54.644123 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":116,"threads_started":1}
I20260810 07:20:54.647409 17446 heartbeater.cc:347] Connected to a master server at 127.1.125.254:36789
I20260810 07:20:54.647722 17446 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:54.648468 17446 heartbeater.cc:511] Master 127.1.125.254:36789 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
I20260810 07:20:54.649883 16929 ts_manager.cc:194] Registered new tserver with Master: 9441e032e003499393e5797cc70cb0bf (127.1.125.194:37551)
I20260810 07:20:54.650713 16929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:57655
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:20:54.667829 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:36789
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:35175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:20:54.786118 17453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:20:54.786444 17453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:20:54.786521 17453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:20:54.790668 17453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:20:54.790769 17453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:20:54.790860 17453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:20:54.796381 17453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:35175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:36789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:20:54.797606 17453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:20:54.798728 17453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:20:54.808817 17460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:20:54.808869 17462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:54.808881 17453 server_base.cc:1034] running on GCE node
W20260810 07:20:54.808832 17459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:20:54.809466 17453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:20:54.810180 17453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:20:54.811374 17453 hybrid_clock.cc:648] HybridClock initialized: now 1786346454811352 us; error 34 us; skew 500 ppm
Aug 10 07:20:54 dist-test-slave-44d2 krb5kdc[16862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:20:54.814227 17453 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:20:54.815100 17453 webserver.cc:460] Webserver started at http://127.1.125.195:42847/ using document root <none> and password file <none>
I20260810 07:20:54.815625 17453 fs_manager.cc:359] Metadata directory not provided
I20260810 07:20:54.815685 17453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:20:54.815850 17453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:20:54.817945 17453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "17190ffa3baa47038c7c1c2f411212c7"
format_stamp: "Formatted at 2026-08-10 07:20:54 on dist-test-slave-44d2"
server_key: "a94313001dc38c44860af436778062ae"
server_key_iv: "c3d0bdc3264a4a7147c9e22ceb8142cd"
server_key_version: "encryptionkey@0"
I20260810 07:20:54.818480 17453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "17190ffa3baa47038c7c1c2f411212c7"
format_stamp: "Formatted at 2026-08-10 07:20:54 on dist-test-slave-44d2"
server_key: "a94313001dc38c44860af436778062ae"
server_key_iv: "c3d0bdc3264a4a7147c9e22ceb8142cd"
server_key_version: "encryptionkey@0"
I20260810 07:20:54.822278 17453 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 07:20:54.824697 17471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:54.826005 17453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:20:54.826138 17453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "17190ffa3baa47038c7c1c2f411212c7"
format_stamp: "Formatted at 2026-08-10 07:20:54 on dist-test-slave-44d2"
server_key: "a94313001dc38c44860af436778062ae"
server_key_iv: "c3d0bdc3264a4a7147c9e22ceb8142cd"
server_key_version: "encryptionkey@0"
I20260810 07:20:54.826252 17453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:20:54.840092 17453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:20:54.843310 17453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:20:54.843513 17453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:20:54.844129 17453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:20:54.845120 17453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:20:54.845170 17453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:54.845234 17453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:20:54.845271 17453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:20:54.854570 17453 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:34179
I20260810 07:20:54.854606 17643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:34179 every 8 connection(s)
I20260810 07:20:54.855466 17453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:20:54 dist-test-slave-44d2 krb5kdc[16862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:20:54.865392 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 17453
I20260810 07:20:54.865525 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:20:54.865839 1527 external_mini_cluster.cc:1428] Setting key 8369392a37e9a66eac20de1c5daa4884
I20260810 07:20:54.869901 16986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.857378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:46709 (local address 127.1.125.254:36789)
0810 07:20:54.857529 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:54.857535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:54.858437 (+ 902us) server_negotiation.cc:407] Connection header received
0810 07:20:54.859754 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:54.859757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:54.859805 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:54.859885 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:54.861127 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.861662 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.862776 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.863006 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.863084 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.865658 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:54.865661 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:54.865663 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:54.865689 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:54.867574 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:54.868219 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:54.868226 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:54.868230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:54.868336 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:54.868764 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:54.868768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:54.868770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:54.868970 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:54.869089 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:54.869607 (+ 518us) server_negotiation.cc:299] Negotiation successful
0810 07:20:54.869740 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:20:54.870785 17650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.857829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:36789 (local address 127.1.125.195:46709)
0810 07:20:54.858281 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:54.858324 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:54.859560 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:54.860055 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:54.860061 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:54.860430 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:20:54.860991 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:54.861005 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.861771 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.861773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:54.862668 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:54.862678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.863143 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.863146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:54.863299 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.863839 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:20:54.863856 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:20:54.865405 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:20:54.867694 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:54.867700 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:54.867717 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:54.868099 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:54.868438 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:20:54.868441 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:20:54.868443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:20:54.868661 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:20:54.869089 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:20:54.869097 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:20:54.869356 (+ 259us) client_negotiation.cc:769] Sending connection context
0810 07:20:54.869551 (+ 195us) client_negotiation.cc:240] Negotiation successful
0810 07:20:54.869914 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":107,"threads_started":1}
I20260810 07:20:54.871896 17645 heartbeater.cc:347] Connected to a master server at 127.1.125.254:36789
I20260810 07:20:54.872164 17645 heartbeater.cc:464] Registering TS with master...
I20260810 07:20:54.872768 17645 heartbeater.cc:511] Master 127.1.125.254:36789 requested a full tablet report, sending...
I20260810 07:20:54.873839 16929 ts_manager.cc:194] Registered new tserver with Master: 17190ffa3baa47038c7c1c2f411212c7 (127.1.125.195:34179)
I20260810 07:20:54.874428 16929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:46709
I20260810 07:20:54.880118 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:20:54.889626 16986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.882135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34340 (local address 127.1.125.254:36789)
0810 07:20:54.882253 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:54.882256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:54.882431 (+ 175us) server_negotiation.cc:407] Connection header received
0810 07:20:54.882533 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:54.882535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:54.882572 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:54.882637 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:20:54.883297 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.883810 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.884900 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.885147 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.885260 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.886645 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:20:54.886649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:20:54.886651 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:20:54.886677 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:20:54.888126 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:54.888574 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:54.888577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:54.888578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:54.888654 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:20:54.888900 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:20:54.888903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:20:54.888904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:20:54.889103 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:20:54.889205 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:54.889411 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 07:20:54.889513 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:20:54.892565 16929 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:20:54.895666 16929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:20:54.910104 17666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.904459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57112 (local address 127.1.125.194:37551)
0810 07:20:54.904867 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:54.904879 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:54.905383 (+ 504us) server_negotiation.cc:407] Connection header received
0810 07:20:54.905480 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:54.905486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:54.905762 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:54.905941 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:54.906259 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.907228 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.908690 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.909243 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.909372 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.909545 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:54.909835 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 07:20:54.909937 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":125,"threads_started":1}
I20260810 07:20:54.910080 17664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.904343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:37551 (local address 127.0.0.1:57112)
0810 07:20:54.905032 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:54.905083 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:54.905229 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:54.905938 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:54.905944 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:54.905966 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:54.906135 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:54.906142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.907359 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.907364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:54.908542 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:54.908552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.909372 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.909376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:54.909632 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.909669 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 07:20:54.909795 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 07:20:54.909924 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":64,"thread_start_us":94,"threads_started":1}
I20260810 07:20:54.911296 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.905139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:34179 (local address 127.0.0.1:46196)
0810 07:20:54.905389 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:54.905403 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:54.905509 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:54.906684 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:54.906687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:54.906705 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:54.906832 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:54.906840 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.908359 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.908362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:54.909604 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:54.909612 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.910829 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.910833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:54.910934 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.910962 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 07:20:54.911042 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 07:20:54.911157 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":45,"thread_start_us":100,"threads_started":1}
I20260810 07:20:54.911809 17668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.905710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60922 (local address 127.1.125.193:43955)
0810 07:20:54.906114 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:54.906135 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:54.906162 (+ 27us) server_negotiation.cc:407] Connection header received
0810 07:20:54.906242 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:54.906247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:54.906442 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:54.906623 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:54.906864 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.908009 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.909568 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.910419 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.910543 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.910684 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:54.911540 (+ 856us) server_negotiation.cc:299] Negotiation successful
0810 07:20:54.911658 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":77,"threads_started":1}
I20260810 07:20:54.911859 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.905777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46196 (local address 127.1.125.195:34179)
0810 07:20:54.906271 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:54.906282 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:54.906301 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:20:54.906368 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:54.906372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:54.906522 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:54.907009 (+ 487us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:54.907183 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.908219 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.909734 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.910696 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.910806 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.910951 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:54.911611 (+ 660us) server_negotiation.cc:299] Negotiation successful
0810 07:20:54.911717 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":184,"threads_started":1}
I20260810 07:20:54.911160 17662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:54.904418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:43955 (local address 127.0.0.1:60922)
0810 07:20:54.905282 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:54.905298 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:54.905379 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:54.906613 (+ 1234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:54.906616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:54.906636 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:54.906758 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:54.906764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.908160 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.908163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:54.909406 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:54.909417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:54.910758 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:54.910764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:54.910864 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:54.910887 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:20:54.910955 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:20:54.911034 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":121,"threads_started":1}
I20260810 07:20:54.913414 17332 tablet_service.cc:1467] Processing CreateTablet for tablet fe34c20e93a447efb1a91c3335cfefa0 (DEFAULT_TABLE table=test-table [id=a175bf9e0a174f1495708a97ede6abaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:54.914655 17332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe34c20e93a447efb1a91c3335cfefa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:54.914435 17539 tablet_service.cc:1467] Processing CreateTablet for tablet fe34c20e93a447efb1a91c3335cfefa0 (DEFAULT_TABLE table=test-table [id=a175bf9e0a174f1495708a97ede6abaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:54.915380 17539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe34c20e93a447efb1a91c3335cfefa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:54.915323 17126 tablet_service.cc:1467] Processing CreateTablet for tablet fe34c20e93a447efb1a91c3335cfefa0 (DEFAULT_TABLE table=test-table [id=a175bf9e0a174f1495708a97ede6abaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:20:54.916275 17126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe34c20e93a447efb1a91c3335cfefa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:20:54.922135 17671 tablet_bootstrap.cc:492] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7: Bootstrap starting.
I20260810 07:20:54.922772 17672 tablet_bootstrap.cc:492] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf: Bootstrap starting.
I20260810 07:20:54.924753 17671 tablet_bootstrap.cc:654] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:54.924860 17673 tablet_bootstrap.cc:492] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d: Bootstrap starting.
I20260810 07:20:54.925130 17672 tablet_bootstrap.cc:654] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:54.925678 17671 log.cc:824] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:54.926000 17672 log.cc:824] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:54.926895 17673 tablet_bootstrap.cc:654] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d: Neither blocks nor log segments found. Creating new log.
I20260810 07:20:54.927582 17671 tablet_bootstrap.cc:492] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7: No bootstrap required, opened a new log
I20260810 07:20:54.927575 17673 log.cc:824] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:20:54.927779 17671 ts_tablet_manager.cc:1397] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 07:20:54.929358 17672 tablet_bootstrap.cc:492] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf: No bootstrap required, opened a new log
I20260810 07:20:54.929368 17673 tablet_bootstrap.cc:492] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d: No bootstrap required, opened a new log
I20260810 07:20:54.929549 17672 ts_tablet_manager.cc:1397] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260810 07:20:54.929570 17673 ts_tablet_manager.cc:1397] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:20:54.931805 17671 raft_consensus.cc:348] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441e032e003499393e5797cc70cb0bf" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 37551 } } peers { permanent_uuid: "b4a18d1fa9fe4130978f759354c5557d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43955 } } peers { permanent_uuid: "17190ffa3baa47038c7c1c2f411212c7" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34179 } }
I20260810 07:20:54.932081 17671 raft_consensus.cc:374] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:54.932150 17671 raft_consensus.cc:729] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17190ffa3baa47038c7c1c2f411212c7, State: Initialized, Role: FOLLOWER
I20260810 07:20:54.933362 17671 consensus_queue.cc:260] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441e032e003499393e5797cc70cb0bf" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 37551 } } peers { permanent_uuid: "b4a18d1fa9fe4130978f759354c5557d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43955 } } peers { permanent_uuid: "17190ffa3baa47038c7c1c2f411212c7" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34179 } }
I20260810 07:20:54.933380 17673 raft_consensus.cc:348] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441e032e003499393e5797cc70cb0bf" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 37551 } } peers { permanent_uuid: "b4a18d1fa9fe4130978f759354c5557d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43955 } } peers { permanent_uuid: "17190ffa3baa47038c7c1c2f411212c7" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34179 } }
I20260810 07:20:54.933651 17673 raft_consensus.cc:374] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:54.933720 17673 raft_consensus.cc:729] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a18d1fa9fe4130978f759354c5557d, State: Initialized, Role: FOLLOWER
I20260810 07:20:54.933576 17672 raft_consensus.cc:348] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441e032e003499393e5797cc70cb0bf" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 37551 } } peers { permanent_uuid: "b4a18d1fa9fe4130978f759354c5557d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43955 } } peers { permanent_uuid: "17190ffa3baa47038c7c1c2f411212c7" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34179 } }
I20260810 07:20:54.933813 17672 raft_consensus.cc:374] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:20:54.933884 17672 raft_consensus.cc:729] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9441e032e003499393e5797cc70cb0bf, State: Initialized, Role: FOLLOWER
I20260810 07:20:54.934238 17645 heartbeater.cc:503] Master 127.1.125.254:36789 was elected leader, sending a full tablet report...
I20260810 07:20:54.934202 17673 consensus_queue.cc:260] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441e032e003499393e5797cc70cb0bf" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 37551 } } peers { permanent_uuid: "b4a18d1fa9fe4130978f759354c5557d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43955 } } peers { permanent_uuid: "17190ffa3baa47038c7c1c2f411212c7" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34179 } }
I20260810 07:20:54.934377 17671 ts_tablet_manager.cc:1428] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260810 07:20:54.934322 17672 consensus_queue.cc:260] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441e032e003499393e5797cc70cb0bf" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 37551 } } peers { permanent_uuid: "b4a18d1fa9fe4130978f759354c5557d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43955 } } peers { permanent_uuid: "17190ffa3baa47038c7c1c2f411212c7" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34179 } }
I20260810 07:20:54.935092 17672 ts_tablet_manager.cc:1428] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:20:54.935164 17673 ts_tablet_manager.cc:1428] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:20:54.935400 17446 heartbeater.cc:503] Master 127.1.125.254:36789 was elected leader, sending a full tablet report...
I20260810 07:20:54.935487 17240 heartbeater.cc:503] Master 127.1.125.254:36789 was elected leader, sending a full tablet report...
I20260810 07:20:55.101821 17680 raft_consensus.cc:482] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:20:55.102075 17680 raft_consensus.cc:504] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441e032e003499393e5797cc70cb0bf" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 37551 } } peers { permanent_uuid: "b4a18d1fa9fe4130978f759354c5557d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43955 } } peers { permanent_uuid: "17190ffa3baa47038c7c1c2f411212c7" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34179 } }
I20260810 07:20:55.103349 17680 leader_election.cc:288] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4a18d1fa9fe4130978f759354c5557d (127.1.125.193:43955), 17190ffa3baa47038c7c1c2f411212c7 (127.1.125.195:34179)
W20260810 07:20:55.106491 17647 tablet.cc:2369] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:20:55.108282 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:55.104254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:53605 (local address 127.1.125.195:34179)
0810 07:20:55.104392 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:55.104397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:55.104412 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:20:55.104470 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:55.104473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:55.104545 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:55.104644 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:55.104983 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:55.105710 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:55.107151 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:55.107562 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:55.107644 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:55.107690 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:55.108042 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 07:20:55.108153 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:20:55.108307 17448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:55.103665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:43955 (local address 127.1.125.194:59551)
0810 07:20:55.103834 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:55.103852 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:55.103966 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:55.104408 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:55.104411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:55.104433 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:55.104605 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:55.104617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:55.105679 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:55.105682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:55.106979 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:55.106997 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:55.107908 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:55.107911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:55.108035 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:55.108056 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:20:55.108120 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:20:55.108191 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":51}
I20260810 07:20:55.108680 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:55.103938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:34179 (local address 127.1.125.194:53605)
0810 07:20:55.104288 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:55.104302 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:20:55.104400 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:20:55.104669 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:20:55.104675 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:20:55.104698 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:20:55.104838 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:55.104845 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:55.105821 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:55.105827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:55.106976 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:20:55.106989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:20:55.107772 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:20:55.107775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:20:55.107888 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:55.107913 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 07:20:55.108434 (+ 521us) client_negotiation.cc:240] Negotiation successful
0810 07:20:55.108543 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":11,"thread_start_us":106,"threads_started":1}
I20260810 07:20:55.108770 17668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:55.103770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:59551 (local address 127.1.125.193:43955)
0810 07:20:55.103911 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:55.103915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:55.103932 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:20:55.104123 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:55.104127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:55.104190 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:55.104269 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:20:55.104756 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:55.105556 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:55.107140 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:55.107779 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:55.107902 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:55.107950 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:55.108553 (+ 603us) server_negotiation.cc:299] Negotiation successful
0810 07:20:55.108644 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:20:55.109395 17571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe34c20e93a447efb1a91c3335cfefa0" candidate_uuid: "9441e032e003499393e5797cc70cb0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17190ffa3baa47038c7c1c2f411212c7" is_pre_election: true
I20260810 07:20:55.109393 17159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe34c20e93a447efb1a91c3335cfefa0" candidate_uuid: "9441e032e003499393e5797cc70cb0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a18d1fa9fe4130978f759354c5557d" is_pre_election: true
I20260810 07:20:55.109762 17571 raft_consensus.cc:2454] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9441e032e003499393e5797cc70cb0bf in term 0.
I20260810 07:20:55.109762 17159 raft_consensus.cc:2454] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9441e032e003499393e5797cc70cb0bf in term 0.
I20260810 07:20:55.110296 17272 leader_election.cc:302] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17190ffa3baa47038c7c1c2f411212c7, 9441e032e003499393e5797cc70cb0bf; no voters:
I20260810 07:20:55.110598 17680 raft_consensus.cc:2781] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:20:55.110694 17680 raft_consensus.cc:482] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:20:55.110749 17680 raft_consensus.cc:3037] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:55.111855 17680 raft_consensus.cc:504] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441e032e003499393e5797cc70cb0bf" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 37551 } } peers { permanent_uuid: "b4a18d1fa9fe4130978f759354c5557d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43955 } } peers { permanent_uuid: "17190ffa3baa47038c7c1c2f411212c7" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34179 } }
I20260810 07:20:55.112282 17680 leader_election.cc:288] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [CANDIDATE]: Term 1 election: Requested vote from peers b4a18d1fa9fe4130978f759354c5557d (127.1.125.193:43955), 17190ffa3baa47038c7c1c2f411212c7 (127.1.125.195:34179)
I20260810 07:20:55.112774 17159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe34c20e93a447efb1a91c3335cfefa0" candidate_uuid: "9441e032e003499393e5797cc70cb0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a18d1fa9fe4130978f759354c5557d"
I20260810 07:20:55.112804 17571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe34c20e93a447efb1a91c3335cfefa0" candidate_uuid: "9441e032e003499393e5797cc70cb0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17190ffa3baa47038c7c1c2f411212c7"
I20260810 07:20:55.112923 17159 raft_consensus.cc:3037] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:55.112934 17571 raft_consensus.cc:3037] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:20:55.114077 17159 raft_consensus.cc:2454] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9441e032e003499393e5797cc70cb0bf in term 1.
I20260810 07:20:55.114300 17571 raft_consensus.cc:2454] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9441e032e003499393e5797cc70cb0bf in term 1.
I20260810 07:20:55.114380 17272 leader_election.cc:302] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9441e032e003499393e5797cc70cb0bf, b4a18d1fa9fe4130978f759354c5557d; no voters:
I20260810 07:20:55.114583 17680 raft_consensus.cc:2781] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:20:55.114849 17680 raft_consensus.cc:686] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [term 1 LEADER]: Becoming Leader. State: Replica: 9441e032e003499393e5797cc70cb0bf, State: Running, Role: LEADER
I20260810 07:20:55.115197 17680 consensus_queue.cc:237] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441e032e003499393e5797cc70cb0bf" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 37551 } } peers { permanent_uuid: "b4a18d1fa9fe4130978f759354c5557d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43955 } } peers { permanent_uuid: "17190ffa3baa47038c7c1c2f411212c7" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34179 } }
I20260810 07:20:55.118809 16925 catalog_manager.cc:5495] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 9441e032e003499393e5797cc70cb0bf (127.1.125.194). New cstate: current_term: 1 leader_uuid: "9441e032e003499393e5797cc70cb0bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441e032e003499393e5797cc70cb0bf" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 37551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4a18d1fa9fe4130978f759354c5557d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17190ffa3baa47038c7c1c2f411212c7" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34179 } health_report { overall_health: UNKNOWN } } }
W20260810 07:20:55.129102 17447 tablet.cc:2369] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:20:55.145325 17666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:55.141370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57114 (local address 127.1.125.194:37551)
0810 07:20:55.141516 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 07:20:55.141521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:20:55.141537 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:20:55.141710 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:20:55.141713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:20:55.141773 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:20:55.141885 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:20:55.142146 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:55.142753 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:55.143691 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:20:55.143917 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:20:55.144025 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:20:55.144382 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:20:55.144878 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:20:55.145012 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 07:20:55.145078 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 07:20:55.145148 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 07:20:55.147100 17241 tablet.cc:2369] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:20:55.151726 17571 raft_consensus.cc:1261] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7 [term 1 FOLLOWER]: Refusing update from remote peer 9441e032e003499393e5797cc70cb0bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:55.151744 17159 raft_consensus.cc:1261] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d [term 1 FOLLOWER]: Refusing update from remote peer 9441e032e003499393e5797cc70cb0bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:20:55.152550 17680 consensus_queue.cc:1035] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "17190ffa3baa47038c7c1c2f411212c7" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:55.152776 17684 consensus_queue.cc:1035] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a18d1fa9fe4130978f759354c5557d" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:20:55.160338 17689 mvcc.cc:205] Tried to move back new op lower bound from 7316875080293982208 to 7316875080154791936. Current Snapshot: MvccSnapshot[applied={T|T < 7316875080293982208}]
I20260810 07:20:55.160777 17690 mvcc.cc:205] Tried to move back new op lower bound from 7316875080293982208 to 7316875080154791936. Current Snapshot: MvccSnapshot[applied={T|T < 7316875080293982208}]
I20260810 07:20:55.170903 16929 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:55.171098 16929 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:20:55.173681 16929 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9f731c54e734957b10913a8a816424f: Sending DeleteTablet for 3 replicas of tablet fe34c20e93a447efb1a91c3335cfefa0
I20260810 07:20:55.174451 17332 tablet_service.cc:1514] Processing DeleteTablet for tablet fe34c20e93a447efb1a91c3335cfefa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:55 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:57112
I20260810 07:20:55.174685 17126 tablet_service.cc:1514] Processing DeleteTablet for tablet fe34c20e93a447efb1a91c3335cfefa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:55 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:60922
I20260810 07:20:55.174770 17539 tablet_service.cc:1514] Processing DeleteTablet for tablet fe34c20e93a447efb1a91c3335cfefa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:20:55 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:46196
I20260810 07:20:55.175213 17702 tablet_replica.cc:330] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d: stopping tablet replica
I20260810 07:20:55.175289 17703 tablet_replica.cc:330] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7: stopping tablet replica
I20260810 07:20:55.175662 17702 raft_consensus.cc:2229] T fe34c20e93a447efb1a91c3335cfefa0 P b4a18d1fa9fe4130978f759354c5557d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:55.175665 17703 raft_consensus.cc:2229] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:20:55.176043 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 17032
I20260810 07:20:55.176294 17703 raft_consensus.cc:2258] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:55.182075 17703 ts_tablet_manager.cc:1905] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:55.186178 17703 ts_tablet_manager.cc:1918] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:55.186311 17703 log.cc:1196] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/fe34c20e93a447efb1a91c3335cfefa0
I20260810 07:20:55.186777 17703 ts_tablet_manager.cc:1939] T fe34c20e93a447efb1a91c3335cfefa0 P 17190ffa3baa47038c7c1c2f411212c7: Deleting consensus metadata
I20260810 07:20:55.188762 16908 catalog_manager.cc:4841] TS 17190ffa3baa47038c7c1c2f411212c7 (127.1.125.195:34179): tablet fe34c20e93a447efb1a91c3335cfefa0 (table test-table [id=a175bf9e0a174f1495708a97ede6abaf]) successfully deleted
I20260810 07:20:55.191425 17700 tablet_replica.cc:330] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf: stopping tablet replica
I20260810 07:20:55.191824 17700 raft_consensus.cc:2229] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:20:55.192323 17700 raft_consensus.cc:2258] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:20:55.194144 17700 ts_tablet_manager.cc:1905] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:20:55.197090 17700 ts_tablet_manager.cc:1918] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:20:55.197211 17700 log.cc:1196] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/fe34c20e93a447efb1a91c3335cfefa0
I20260810 07:20:55.197613 17700 ts_tablet_manager.cc:1939] T fe34c20e93a447efb1a91c3335cfefa0 P 9441e032e003499393e5797cc70cb0bf: Deleting consensus metadata
W20260810 07:20:55.198046 16908 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:43955 (error 108)
I20260810 07:20:55.198890 16905 catalog_manager.cc:4841] TS 9441e032e003499393e5797cc70cb0bf (127.1.125.194:37551): tablet fe34c20e93a447efb1a91c3335cfefa0 (table test-table [id=a175bf9e0a174f1495708a97ede6abaf]) successfully deleted
I20260810 07:20:55.199213 17662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:20:55.198883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:43955 (local address 127.0.0.1:60934)
0810 07:20:55.199029 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 07:20:55.199116 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 07:20:55.199355 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 17247
W20260810 07:20:55.199431 16908 catalog_manager.cc:4584] TS b4a18d1fa9fe4130978f759354c5557d (127.1.125.193:43955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fe34c20e93a447efb1a91c3335cfefa0: Network error: Client connection negotiation failed: client connection to 127.1.125.193:43955: connect: Connection refused (error 111)
I20260810 07:20:55.214462 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 17453
I20260810 07:20:55.228471 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 16884
2026-08-10T07:20:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2715 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:20:55 dist-test-slave-44d2 krb5kdc[17715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:20:55 dist-test-slave-44d2 krb5kdc[17715](info): set up 2 sockets
Aug 10 07:20:55 dist-test-slave-44d2 krb5kdc[17715](info): commencing operation
krb5kdc: starting...
W20260810 07:20:56.498973 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:20:56 dist-test-slave-44d2 krb5kdc[17715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:20:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:20:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/rangeradmin_127.1.125.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/rangeradmin_127.1.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/rangerlookup_127.1.125.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/rangerlookup_127.1.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/HTTP_127.1.125.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/HTTP_127.1.125.212@KRBTEST.COM.keytab.
I20260810 07:20:56.687335 1527 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwKbuEn/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/postgres -l logfile start
2026-08-10 07:20:59.313 UTC [17764] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 07:20:59.313 UTC [17764] LOG: listening on IPv4 address "127.1.125.212", port 32849
2026-08-10 07:20:59.316 UTC [17764] LOG: listening on Unix socket "/tmp/.s.PGSQL.32849"
2026-08-10 07:20:59.338 UTC [17771] LOG: database system was shut down at 2026-08-10 07:20:57 UTC
2026-08-10 07:20:59.342 UTC [17764] LOG: database system is ready to accept connections
I20260810 07:21:00.497884 1527 mini_postgres.cc:96] Postgres bound to 32849
2026-08-10 07:21:00.508 UTC [17781] FATAL: database "slave" does not exist
127.1.125.212:32849 - accepting connections
I20260810 07:21:00.509436 1527 mini_ranger.cc:161] Starting Ranger...
I20260810 07:21:00.527861 1527 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 07:21:01.017113 1527 mini_ranger.cc:87] Created ranger Postgres database
I20260810 07:21:01.017223 1527 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-admin
2026-08-10 07:21:01,424 [I] DB FLAVOR :POSTGRES
2026-08-10 07:21:01,424 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:21:01,424 [I] Checking connection..
2026-08-10 07:21:01,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 07:21:01,688 [I] Checking connection passed.
2026-08-10 07:21:01,688 [I] --------- Verifying version history table ---------
2026-08-10 07:21:01,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:21:01,949 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:21:01,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:21:02,216 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:21:02,217 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 07:21:02,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 07:21:02.469 UTC [17886] WARNING: there is no transaction in progress
2026-08-10 07:21:02,479 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 07:21:02,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:21:02,750 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 07:21:02,751 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:21:02,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 07:21:03,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 07:21:03,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-44d2.c.gcp-upstream.internal','N') ;"
2026-08-10 07:21:03,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:21:03,777 [I] Table x_portal_user does not exist in database ranger
2026-08-10 07:21:03,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:21:04,156 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 07:21:04,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:21:04,642 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:21:05,080 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 07:21:05,080 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 07:21:06.067 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.083 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.102 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.118 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.134 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.151 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.209 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.217 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.233 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.243 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.254 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.268 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.278 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.285 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.292 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06.298 UTC [18045] WARNING: there is no transaction in progress
2026-08-10 07:21:06,972 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 07:21:06,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 07:21:07,239 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 07:21:07,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:21:07,516 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 07:21:07,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:21:07,758 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 07:21:07,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:21:08,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:21:08,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-44d2.c.gcp-upstream.internal';"
2026-08-10 07:21:08,516 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 07:21:08,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 07:21:08,792 [I] DB_PATCHES have already been applied
I20260810 07:21:08.799131 1527 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwKbuEn/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 07:21:08.799214 1527 mini_ranger.cc:193] Using host: 127.1.125.212
I20260810 07:21:08.804311 1527 mini_ranger.cc:227] Ranger admin URL: http://127.1.125.212:54417
Aug 10, 2026 7:21:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:21:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:21:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 7:21:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 7:21:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 7:21:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 7:21:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 07:21:10.137949 1527 mini_ranger.cc:160] Time spent starting Ranger: real 9.629s user 0.003s sys 0.010s
Aug 10, 2026 7:21:31 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 7:21:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 7:21:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 7:21:31 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 7:21:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 7:21:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 7:21:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 7:21:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 7:21:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 7:21:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 7:21:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 7:21:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 7:21:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 7:21:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 7:21:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 7:21:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 7:21:31 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 7:21:32 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 07:21:33.427445 1527 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/rangerkms_127.1.125.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/rangerkms_127.1.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.125.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.125.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.125.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/HTTP_127.1.125.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.125.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/HTTP_127.1.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 07:21:33 dist-test-slave-44d2 krb5kdc[17715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346493, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 07:21:33.522111 1527 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 07:21:33.533275 1527 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 07:21:34.431612 1527 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 07:21:34.431701 1527 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-kms
I20260810 07:21:34.431795 1527 mini_ranger_kms.cc:227] Using postgres at 127.1.125.212:32849
2026-08-10 07:21:34,746 [I] DB FLAVOR :POSTGRES
2026-08-10 07:21:34,746 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:21:34,747 [I] Checking connection
2026-08-10 07:21:34,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 07:21:35,020 [I] connection success
2026-08-10 07:21:35,020 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 07:21:35,020 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 07:21:35,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 07:21:35,274 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 07:21:35,274 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:21:35,274 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 07:21:35,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:32849/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 07:21:35,551 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 07:21:36.113281 1527 mini_ranger_kms.cc:310] Created kms service
I20260810 07:21:36.333269 1527 mini_ranger_kms.cc:326] Created kudu user
I20260810 07:21:36.381847 1527 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 07:21:36 dist-test-slave-44d2 krb5kdc[17715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346493, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:21:36.832199 1527 mini_ranger_kms.cc:384] Added ranger policy
I20260810 07:21:36.832366 1527 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 07:21:36.832388 1527 mini_ranger_kms.cc:240] Using host: 127.1.125.212
I20260810 07:21:36.838678 1527 mini_ranger_kms.cc:276] Ranger KMS PID: 18330
I20260810 07:21:36.838820 1527 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.125.212:36501
Aug 10, 2026 7:21:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:21:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:21:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 7:21:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 7:21:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 7:21:38 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 07:21:39 dist-test-slave-44d2 krb5kdc[17715](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346499, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.125.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:21:39.653053 1527 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.131s user 0.033s sys 0.265s
I20260810 07:21:39.653174 1527 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 07:21:39.653316 1527 mini_ranger_kms.cc:401] 127.1.125.212:36501/kms/v1/keys
Aug 10 07:21:40 dist-test-slave-44d2 krb5kdc[17715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346499, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.125.212@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
Aug 10 07:21:40 dist-test-slave-44d2 krb5kdc[17715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346499, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.125.212@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
Aug 10 07:21:42 dist-test-slave-44d2 krb5kdc[17715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:21:42.832893 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:39007
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:33521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:39007
--ranger_config_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.125.212:36501/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:21:42.954198 18402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:21:42.954507 18402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:21:42.954593 18402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:21:42.958737 18402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:21:42.958833 18402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:21:42.958881 18402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:21:42.958937 18402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:21:42.958976 18402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:21:42.964674 18402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:33521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.125.212:36501/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:39007
--ranger_config_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:39007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:21:42.965795 18402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:21:42.966892 18402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:21:42.976272 18411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:21:42.976553 18410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:21:42.976293 18413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:21:42.976420 18402 server_base.cc:1034] running on GCE node
I20260810 07:21:42.977273 18402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:21:42.978471 18402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:21:42.979673 18402 hybrid_clock.cc:648] HybridClock initialized: now 1786346502979647 us; error 48 us; skew 500 ppm
Aug 10 07:21:42 dist-test-slave-44d2 krb5kdc[17715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346502, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:21:42.987201 18402 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:21:42.988374 18402 webserver.cc:460] Webserver started at http://127.1.125.254:46169/ using document root <none> and password file <none>
I20260810 07:21:42.989025 18402 fs_manager.cc:359] Metadata directory not provided
I20260810 07:21:42.989092 18402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:21:42.989337 18402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:21:42 dist-test-slave-44d2 krb5kdc[17715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346502, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:21:43.045300 18402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "0fa3e9dc1c134fafa50870a5a40f0641"
format_stamp: "Formatted at 2026-08-10 07:21:43 on dist-test-slave-44d2"
server_key: "e044b8c06d490452b54fe8f66eb516aa"
server_key_iv: "7fab42546c61d45764e11befbd5add58"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:43.046142 18402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "0fa3e9dc1c134fafa50870a5a40f0641"
format_stamp: "Formatted at 2026-08-10 07:21:43 on dist-test-slave-44d2"
server_key: "e044b8c06d490452b54fe8f66eb516aa"
server_key_iv: "7fab42546c61d45764e11befbd5add58"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:43.050479 18402 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:21:43.065332 18424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:43.066884 18402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:21:43.067044 18402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "0fa3e9dc1c134fafa50870a5a40f0641"
format_stamp: "Formatted at 2026-08-10 07:21:43 on dist-test-slave-44d2"
server_key: "e044b8c06d490452b54fe8f66eb516aa"
server_key_iv: "7fab42546c61d45764e11befbd5add58"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:43.067203 18402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:21:43.093657 18402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:21:43.096895 18402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:21:43.097129 18402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:21:43.105880 18402 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:39007
I20260810 07:21:43.105921 18502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:39007 every 8 connection(s)
I20260810 07:21:43.106936 18402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:21:43.110030 18503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:21:43.112823 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 18402
I20260810 07:21:43.112959 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:21:43.113266 1527 external_mini_cluster.cc:1428] Setting key ca6e92ea47632e789f65c2dc449f3c80
I20260810 07:21:43.116719 18503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641: Bootstrap starting.
I20260810 07:21:43.119530 18503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641: Neither blocks nor log segments found. Creating new log.
I20260810 07:21:43.120390 18503 log.cc:824] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641: Log is configured to *not* fsync() on all Append() calls
I20260810 07:21:43.122763 18503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641: No bootstrap required, opened a new log
Aug 10 07:21:43 dist-test-slave-44d2 krb5kdc[17715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:21:43.127300 18503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa3e9dc1c134fafa50870a5a40f0641" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39007 } }
I20260810 07:21:43.127570 18503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:21:43.127683 18503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa3e9dc1c134fafa50870a5a40f0641, State: Initialized, Role: FOLLOWER
I20260810 07:21:43.128196 18503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa3e9dc1c134fafa50870a5a40f0641" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39007 } }
I20260810 07:21:43.128363 18503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:21:43.128449 18503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:21:43.128628 18503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:21:43.129678 18503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa3e9dc1c134fafa50870a5a40f0641" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39007 } }
I20260810 07:21:43.130067 18503 leader_election.cc:302] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fa3e9dc1c134fafa50870a5a40f0641; no voters:
I20260810 07:21:43.130461 18503 leader_election.cc:288] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:21:43.130534 18512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:21:43.130578 18510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:43.114823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47386 (local address 127.1.125.254:39007)
0810 07:21:43.115328 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:43.115351 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:43.115397 (+ 46us) server_negotiation.cc:407] Connection header received
0810 07:21:43.116234 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:43.116258 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:43.116743 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:43.117135 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:21:43.118007 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:43.119491 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:43.120779 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:43.121350 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:43.121522 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:43.124717 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:21:43.124724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:21:43.124741 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:21:43.124773 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:21:43.126789 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:43.127323 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:43.127329 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:43.127336 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:43.127472 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:43.127815 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:43.127818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:43.127819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:43.128309 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:21:43.129118 (+ 809us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:43.129267 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 07:21:43.129579 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":135,"threads_started":1}
I20260810 07:21:43.131179 18512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [term 1 LEADER]: Becoming Leader. State: Replica: 0fa3e9dc1c134fafa50870a5a40f0641, State: Running, Role: LEADER
I20260810 07:21:43.131554 18512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa3e9dc1c134fafa50870a5a40f0641" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39007 } }
I20260810 07:21:43.131899 18503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:21:43.132964 18512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fa3e9dc1c134fafa50870a5a40f0641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa3e9dc1c134fafa50870a5a40f0641" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39007 } } }
I20260810 07:21:43.133450 18512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [sys.catalog]: This master's current role is: LEADER
I20260810 07:21:43.133708 18512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fa3e9dc1c134fafa50870a5a40f0641. Latest consensus state: current_term: 1 leader_uuid: "0fa3e9dc1c134fafa50870a5a40f0641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa3e9dc1c134fafa50870a5a40f0641" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 39007 } } }
I20260810 07:21:43.133879 18531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:21:43.133885 18512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641 [sys.catalog]: This master's current role is: LEADER
I20260810 07:21:43.134529 18503 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 07:21:43.141139 18531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:21:43.150701 18531 catalog_manager.cc:1347] Generated new cluster ID: 6804e3160d324920a4997235e056bfea
I20260810 07:21:43.150794 18531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:21:43.178699 18531 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:21:43.180124 18531 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:21:43.188668 18531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641: Generated new TSK 0
I20260810 07:21:43.189628 18531 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:21:43.928514 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.125.212:36501/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39007
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:33521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:21:44.072814 18584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:21:44.073068 18584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:21:44.073112 18584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:21:44.077256 18584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:21:44.077312 18584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:21:44.077369 18584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:21:44.082687 18584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:33521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.125.212:36501/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:21:44.083791 18584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:21:44.084951 18584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:21:44.096019 18591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:21:44.096103 18594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:21:44.096601 18590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:21:44.096647 18584 server_base.cc:1034] running on GCE node
I20260810 07:21:44.097236 18584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:21:44.098121 18584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:21:44.105592 18584 hybrid_clock.cc:648] HybridClock initialized: now 1786346504105536 us; error 60 us; skew 500 ppm
Aug 10 07:21:44 dist-test-slave-44d2 krb5kdc[17715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:21:44.109656 18584 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:21:44.110931 18584 webserver.cc:460] Webserver started at http://127.1.125.193:42227/ using document root <none> and password file <none>
I20260810 07:21:44.111795 18584 fs_manager.cc:359] Metadata directory not provided
I20260810 07:21:44.111955 18584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:21:44.112303 18584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:21:44 dist-test-slave-44d2 krb5kdc[17715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:21:44.129086 18584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "44e218bfc262445f95be7af4a23564fa"
format_stamp: "Formatted at 2026-08-10 07:21:44 on dist-test-slave-44d2"
server_key: "ec8169a38dc4514e71a1cb9375cc8626"
server_key_iv: "0014587e8bc23d704727ccfd3860b6d5"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:44.129907 18584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "44e218bfc262445f95be7af4a23564fa"
format_stamp: "Formatted at 2026-08-10 07:21:44 on dist-test-slave-44d2"
server_key: "ec8169a38dc4514e71a1cb9375cc8626"
server_key_iv: "0014587e8bc23d704727ccfd3860b6d5"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:44.135433 18584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 07:21:44.155560 18608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:44.157416 18584 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260810 07:21:44.157598 18584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "44e218bfc262445f95be7af4a23564fa"
format_stamp: "Formatted at 2026-08-10 07:21:44 on dist-test-slave-44d2"
server_key: "ec8169a38dc4514e71a1cb9375cc8626"
server_key_iv: "0014587e8bc23d704727ccfd3860b6d5"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:44.157756 18584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:21:44.179391 18584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:21:44.183326 18584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:21:44.183614 18584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:21:44.184419 18584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:21:44.185586 18584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:21:44.185689 18584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:44.185779 18584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:21:44.185833 18584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:44.197113 18584 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:37855
I20260810 07:21:44.198382 18584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:21:44.207293 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 18584
I20260810 07:21:44.207455 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:21:44.207834 1527 external_mini_cluster.cc:1428] Setting key c6ab4389a7ee7b645b8be1b95fe6ac0c
I20260810 07:21:44.208976 18782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:37855 every 8 connection(s)
Aug 10 07:21:44 dist-test-slave-44d2 krb5kdc[17715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:21:44.229065 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:44.212827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:46185 (local address 127.1.125.254:39007)
0810 07:21:44.213323 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:44.213330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:44.214229 (+ 899us) server_negotiation.cc:407] Connection header received
0810 07:21:44.215601 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:44.215604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:44.215666 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:44.215753 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:21:44.217436 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:44.218334 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:44.220275 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:44.220666 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:44.220766 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:44.224005 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:21:44.224009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:21:44.224013 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:21:44.224055 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:21:44.225929 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:44.226584 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:44.226587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:44.226589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:44.226696 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:44.227182 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:44.227188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:44.227190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:44.227418 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:21:44.227542 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:44.228651 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0810 07:21:44.228815 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":156,"threads_started":1}
I20260810 07:21:44.231055 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:44.211707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39007 (local address 127.1.125.193:46185)
0810 07:21:44.212274 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:44.214084 (+ 1810us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:44.215172 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:44.216126 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:44.216135 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:44.216540 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:21:44.217240 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:44.217260 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:44.218507 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:44.218511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:44.220092 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:44.220104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:44.220955 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:44.220959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:44.221166 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:44.221895 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:21:44.221919 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:21:44.223798 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:21:44.226086 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:21:44.226092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:21:44.226109 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:21:44.226464 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:21:44.226821 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:21:44.226824 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:21:44.226826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:21:44.227049 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:21:44.227603 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:21:44.227610 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:21:44.228052 (+ 442us) client_negotiation.cc:769] Sending connection context
0810 07:21:44.228269 (+ 217us) client_negotiation.cc:240] Negotiation successful
0810 07:21:44.229766 (+ 1497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":165,"threads_started":1}
I20260810 07:21:44.233033 18784 heartbeater.cc:347] Connected to a master server at 127.1.125.254:39007
I20260810 07:21:44.233302 18784 heartbeater.cc:464] Registering TS with master...
I20260810 07:21:44.233906 18784 heartbeater.cc:511] Master 127.1.125.254:39007 requested a full tablet report, sending...
I20260810 07:21:44.235780 18446 ts_manager.cc:194] Registered new tserver with Master: 44e218bfc262445f95be7af4a23564fa (127.1.125.193:37855)
I20260810 07:21:44.237406 18446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:46185
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:21:44.263810 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.125.212:36501/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39007
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:33521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:21:44 dist-test-slave-44d2 krb5kdc[17715](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 07:21:44.396807 18798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:21:44.397095 18798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:21:44.397209 18798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:21:44.401453 18798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:21:44.401578 18798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:21:44.401685 18798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:21:44.407753 18798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:33521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.125.212:36501/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:21:44.409066 18798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:21:44.410104 18798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:21:44.420049 18810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:21:44.420049 18815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:21:44.420511 18811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:21:44.421082 18798 server_base.cc:1034] running on GCE node
I20260810 07:21:44.421489 18798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:21:44.422232 18798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:21:44.423449 18798 hybrid_clock.cc:648] HybridClock initialized: now 1786346504423413 us; error 47 us; skew 500 ppm
Aug 10 07:21:44 dist-test-slave-44d2 krb5kdc[17715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:21:44.426426 18798 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:21:44.427423 18798 webserver.cc:460] Webserver started at http://127.1.125.194:45939/ using document root <none> and password file <none>
I20260810 07:21:44.428006 18798 fs_manager.cc:359] Metadata directory not provided
I20260810 07:21:44.428097 18798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:21:44.428318 18798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:21:44 dist-test-slave-44d2 krb5kdc[17715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:21:44.445466 18798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "5de4d22d341445ccb52f9093ee81623a"
format_stamp: "Formatted at 2026-08-10 07:21:44 on dist-test-slave-44d2"
server_key: "8f8c2574a47bc3f071a2ce7ef316a601"
server_key_iv: "8bd1b6aa9e2aa1f35a55214bd2948844"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:44.446141 18798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "5de4d22d341445ccb52f9093ee81623a"
format_stamp: "Formatted at 2026-08-10 07:21:44 on dist-test-slave-44d2"
server_key: "8f8c2574a47bc3f071a2ce7ef316a601"
server_key_iv: "8bd1b6aa9e2aa1f35a55214bd2948844"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:44.456548 18798 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.001s
I20260810 07:21:44.492916 18825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:44.494781 18798 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260810 07:21:44.495005 18798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "5de4d22d341445ccb52f9093ee81623a"
format_stamp: "Formatted at 2026-08-10 07:21:44 on dist-test-slave-44d2"
server_key: "8f8c2574a47bc3f071a2ce7ef316a601"
server_key_iv: "8bd1b6aa9e2aa1f35a55214bd2948844"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:44.495234 18798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:21:44.543756 18798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:21:44.547825 18798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:21:44.548056 18798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:21:44.548892 18798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:21:44.550040 18798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:21:44.550122 18798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:44.550182 18798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:21:44.550279 18798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:44.562610 18798 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:46443
I20260810 07:21:44.563755 18798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:21:44.565349 19024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:46443 every 8 connection(s)
I20260810 07:21:44.574091 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 18798
I20260810 07:21:44.574240 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:21:44.574602 1527 external_mini_cluster.cc:1428] Setting key a5a60f5e8e51e9da5b88e454d93c8c2b
Aug 10 07:21:44 dist-test-slave-44d2 krb5kdc[17715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:21:44.579759 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:44.567487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:55637 (local address 127.1.125.254:39007)
0810 07:21:44.567613 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:44.567618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:44.567635 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:21:44.568356 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:44.568360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:44.568425 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:44.568537 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:21:44.569820 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:44.570379 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:44.572049 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:44.572339 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:44.572426 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:44.575327 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:21:44.575331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:21:44.575334 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:21:44.575362 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:21:44.577247 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:44.577871 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:44.577878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:44.577882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:44.577982 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:44.578396 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:44.578402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:44.578406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:44.578664 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:21:44.578774 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:44.579420 (+ 646us) server_negotiation.cc:299] Negotiation successful
0810 07:21:44.579556 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:21:44.580749 19032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:44.566382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39007 (local address 127.1.125.194:55637)
0810 07:21:44.566894 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:44.566949 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:44.568119 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:44.568737 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:44.568745 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:44.569085 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:21:44.569587 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:44.569601 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:44.570571 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:44.570575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:44.571925 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:44.571938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:44.572537 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:44.572543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:44.572765 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:44.573386 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:21:44.573408 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:21:44.575128 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:21:44.577381 (+ 2253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:21:44.577388 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:21:44.577404 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:21:44.577756 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:21:44.578090 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:21:44.578093 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:21:44.578095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:21:44.578285 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:21:44.578851 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:21:44.578861 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:21:44.579167 (+ 306us) client_negotiation.cc:769] Sending connection context
0810 07:21:44.579373 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 07:21:44.579709 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":153,"threads_started":1}
I20260810 07:21:44.582023 19028 heartbeater.cc:347] Connected to a master server at 127.1.125.254:39007
I20260810 07:21:44.582304 19028 heartbeater.cc:464] Registering TS with master...
I20260810 07:21:44.582826 19028 heartbeater.cc:511] Master 127.1.125.254:39007 requested a full tablet report, sending...
I20260810 07:21:44.583961 18446 ts_manager.cc:194] Registered new tserver with Master: 5de4d22d341445ccb52f9093ee81623a (127.1.125.194:46443)
I20260810 07:21:44.584632 18446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:55637
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:21:44.618467 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.125.212:36501/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39007
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:33521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:21:44 dist-test-slave-44d2 krb5kdc[17715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
W20260810 07:21:44.780568 19037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:21:44.781018 19037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:21:44.781137 19037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:21:44.787204 19037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:21:44.787364 19037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:21:44.787528 19037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:21:44.795545 19037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:33521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.125.212:36501/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:39007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:21:44.797219 19037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:21:44.798663 19037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:21:44.812093 19049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:21:44.812754 19044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:21:44.813418 19037 server_base.cc:1034] running on GCE node
W20260810 07:21:44.821589 19042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:21:44.822321 19037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:21:44.823238 19037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:21:44.824664 19037 hybrid_clock.cc:648] HybridClock initialized: now 1786346504824591 us; error 94 us; skew 500 ppm
Aug 10 07:21:44 dist-test-slave-44d2 krb5kdc[17715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:21:44.828805 19037 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:21:44.830231 19037 webserver.cc:460] Webserver started at http://127.1.125.195:44903/ using document root <none> and password file <none>
I20260810 07:21:44.831156 19037 fs_manager.cc:359] Metadata directory not provided
I20260810 07:21:44.831336 19037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:21:44.831719 19037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:21:44 dist-test-slave-44d2 krb5kdc[17715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:21:44.878319 19037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "2f9797f0d02b44dc95eaebd748b0b344"
format_stamp: "Formatted at 2026-08-10 07:21:44 on dist-test-slave-44d2"
server_key: "6c04fde643a6e468e7c1cd4e5db469c9"
server_key_iv: "a6492384235b2f14b758945a4e3c5702"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:44.879544 19037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "2f9797f0d02b44dc95eaebd748b0b344"
format_stamp: "Formatted at 2026-08-10 07:21:44 on dist-test-slave-44d2"
server_key: "6c04fde643a6e468e7c1cd4e5db469c9"
server_key_iv: "a6492384235b2f14b758945a4e3c5702"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:44.894376 19037 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.006s sys 0.000s
I20260810 07:21:44.921695 19058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:44.923575 19037 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260810 07:21:44.923729 19037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "2f9797f0d02b44dc95eaebd748b0b344"
format_stamp: "Formatted at 2026-08-10 07:21:44 on dist-test-slave-44d2"
server_key: "6c04fde643a6e468e7c1cd4e5db469c9"
server_key_iv: "a6492384235b2f14b758945a4e3c5702"
server_key_version: "kuduclusterkey@0"
I20260810 07:21:44.923869 19037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:21:44.962909 19037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:21:44.967484 19037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:21:44.967782 19037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:21:44.968621 19037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:21:44.969826 19037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:21:44.969954 19037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:44.970075 19037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:21:44.970158 19037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:21:44.986406 19037 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:39709
I20260810 07:21:44.987804 19037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:21:44.989280 19240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:39709 every 8 connection(s)
I20260810 07:21:44.996662 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 19037
I20260810 07:21:44.996815 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:21:44.997206 1527 external_mini_cluster.cc:1428] Setting key 462ed7cc698cce42cdebe764779e43e3
Aug 10 07:21:44 dist-test-slave-44d2 krb5kdc[17715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:21:45.010100 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:44.993214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:52717 (local address 127.1.125.254:39007)
0810 07:21:44.993353 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:44.993359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:44.994315 (+ 956us) server_negotiation.cc:407] Connection header received
0810 07:21:44.995794 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:44.995798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:44.995859 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:44.995930 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:21:44.997727 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:44.998638 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.000642 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.001052 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.001133 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.004510 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:21:45.004515 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:21:45.004518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:21:45.004546 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:21:45.006826 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:45.008103 (+ 1277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:45.008107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:45.008109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:45.008205 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:45.008685 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:45.008689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:45.008691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:45.008940 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:21:45.009031 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:45.009751 (+ 720us) server_negotiation.cc:299] Negotiation successful
0810 07:21:45.009913 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:21:45.012125 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:44.991980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:39007 (local address 127.1.125.195:52717)
0810 07:21:44.994140 (+ 2160us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:44.994197 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:44.995515 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:44.996374 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:44.996384 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:44.996826 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:21:44.997542 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:44.997562 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:44.998825 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:44.998829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:45.000452 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:45.000464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.001351 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.001355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:45.001565 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.002354 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:21:45.002378 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:21:45.004275 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:21:45.007569 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:21:45.007575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:21:45.007592 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:21:45.007970 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:21:45.008332 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:21:45.008335 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:21:45.008337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:21:45.008564 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:21:45.009110 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:21:45.009117 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:21:45.009438 (+ 321us) client_negotiation.cc:769] Sending connection context
0810 07:21:45.010619 (+ 1181us) client_negotiation.cc:240] Negotiation successful
0810 07:21:45.011002 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1925,"thread_start_us":150,"threads_started":1}
I20260810 07:21:45.013795 19243 heartbeater.cc:347] Connected to a master server at 127.1.125.254:39007
I20260810 07:21:45.014091 19243 heartbeater.cc:464] Registering TS with master...
I20260810 07:21:45.014731 19243 heartbeater.cc:511] Master 127.1.125.254:39007 requested a full tablet report, sending...
I20260810 07:21:45.015996 18446 ts_manager.cc:194] Registered new tserver with Master: 2f9797f0d02b44dc95eaebd748b0b344 (127.1.125.195:39709)
I20260810 07:21:45.016860 18446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:52717
I20260810 07:21:45.023599 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:21:45.038290 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:45.026391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.1.125.254:39007)
0810 07:21:45.026526 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:45.026531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:45.027336 (+ 805us) server_negotiation.cc:407] Connection header received
0810 07:21:45.027546 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:45.027550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:45.027604 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:45.027661 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:21:45.028944 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.029888 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.031568 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.031943 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.032023 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.034149 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:21:45.034154 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:21:45.034157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:21:45.034186 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:21:45.036198 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:45.036775 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:45.036779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:45.036781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:45.036886 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:21:45.037234 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:21:45.037237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:21:45.037239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:21:45.037468 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:21:45.037590 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:45.037975 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 07:21:45.038105 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:21:45.042625 18446 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:21:45.047677 18446 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:21:45.076689 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:45.062760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:39709 (local address 127.0.0.1:55760)
0810 07:21:45.066371 (+ 3611us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:45.066388 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:45.066498 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:45.068434 (+ 1936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:45.068441 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:45.068469 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:21:45.068645 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:45.068653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.071650 (+ 2997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.071654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:45.073391 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:45.073400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.076042 (+ 2642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.076046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:45.076182 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.076228 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 07:21:45.076345 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 07:21:45.076506 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3541,"thread_start_us":121,"threads_started":1}
I20260810 07:21:45.077976 19260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:45.060543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:37855 (local address 127.0.0.1:44646)
0810 07:21:45.063940 (+ 3397us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:45.064002 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:45.064153 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:45.073349 (+ 9196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:45.073353 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:45.073386 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:21:45.073517 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:45.073524 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.074967 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.074971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:45.076263 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:45.076272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.077508 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.077512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:45.077627 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.077670 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 07:21:45.077753 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 07:21:45.077825 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3170,"thread_start_us":124,"threads_started":1}
I20260810 07:21:45.081782 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:45.060691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44646 (local address 127.1.125.193:37855)
0810 07:21:45.072764 (+ 12073us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:45.072773 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:45.072807 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:21:45.072905 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:45.072912 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:45.073173 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:45.073397 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:21:45.073633 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.074835 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.076402 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.077353 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.077511 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.077681 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:45.081459 (+ 3778us) server_negotiation.cc:299] Negotiation successful
0810 07:21:45.081599 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11924,"thread_start_us":107,"threads_started":1}
I20260810 07:21:45.085803 18681 tablet_service.cc:1467] Processing CreateTablet for tablet 5bc0eb07cdec427d8ca52756e6a79f36 (DEFAULT_TABLE table=test-table [id=d90f7c0bb98f4fa38a4e65ea4c6ce580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:21:45.087249 18681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bc0eb07cdec427d8ca52756e6a79f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:21:45.093959 19264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:45.063243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55760 (local address 127.1.125.195:39709)
0810 07:21:45.067659 (+ 4416us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:45.067674 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:45.067704 (+ 30us) server_negotiation.cc:407] Connection header received
0810 07:21:45.067791 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:45.067797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:45.068059 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:45.068234 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:21:45.068800 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.071414 (+ 2614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.074758 (+ 3344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.075693 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.075808 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.075977 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:45.093608 (+ 17631us) server_negotiation.cc:299] Negotiation successful
0810 07:21:45.093766 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4286,"thread_start_us":136,"threads_started":1}
I20260810 07:21:45.097518 19134 tablet_service.cc:1467] Processing CreateTablet for tablet 5bc0eb07cdec427d8ca52756e6a79f36 (DEFAULT_TABLE table=test-table [id=d90f7c0bb98f4fa38a4e65ea4c6ce580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:21:45.100154 19134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bc0eb07cdec427d8ca52756e6a79f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:21:45.110077 19266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:45.065102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:46443 (local address 127.0.0.1:55822)
0810 07:21:45.070950 (+ 5848us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:45.070966 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:45.071061 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:45.077906 (+ 6845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:45.077909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:45.077928 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:21:45.078019 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:45.078028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.080539 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.080543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:45.093361 (+ 12818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:45.093381 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.109557 (+ 16176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.109562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:45.109684 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.109715 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 07:21:45.109799 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 07:21:45.109912 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5778,"thread_start_us":96,"threads_started":1}
I20260810 07:21:45.112944 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:45.065630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55822 (local address 127.1.125.194:46443)
0810 07:21:45.073691 (+ 8061us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:45.073710 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:45.073735 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:21:45.073812 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:45.073817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:45.077539 (+ 3722us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:45.077721 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:21:45.078181 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.080358 (+ 2177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.107912 (+ 27554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.109172 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.109305 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.109475 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:45.112624 (+ 3149us) server_negotiation.cc:299] Negotiation successful
0810 07:21:45.112774 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7931,"thread_start_us":80,"threads_started":1}
I20260810 07:21:45.116544 18909 tablet_service.cc:1467] Processing CreateTablet for tablet 5bc0eb07cdec427d8ca52756e6a79f36 (DEFAULT_TABLE table=test-table [id=d90f7c0bb98f4fa38a4e65ea4c6ce580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:21:45.117995 18909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bc0eb07cdec427d8ca52756e6a79f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:21:45.127247 19268 tablet_bootstrap.cc:492] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa: Bootstrap starting.
I20260810 07:21:45.132252 19268 tablet_bootstrap.cc:654] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa: Neither blocks nor log segments found. Creating new log.
I20260810 07:21:45.134047 19268 log.cc:824] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa: Log is configured to *not* fsync() on all Append() calls
I20260810 07:21:45.140561 19268 tablet_bootstrap.cc:492] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa: No bootstrap required, opened a new log
I20260810 07:21:45.140805 19268 ts_tablet_manager.cc:1397] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.003s
I20260810 07:21:45.141024 19270 tablet_bootstrap.cc:492] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344: Bootstrap starting.
I20260810 07:21:45.145155 19270 tablet_bootstrap.cc:654] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344: Neither blocks nor log segments found. Creating new log.
I20260810 07:21:45.145656 19268 raft_consensus.cc:348] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44e218bfc262445f95be7af4a23564fa" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37855 } } peers { permanent_uuid: "2f9797f0d02b44dc95eaebd748b0b344" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39709 } } peers { permanent_uuid: "5de4d22d341445ccb52f9093ee81623a" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46443 } }
I20260810 07:21:45.146088 19268 raft_consensus.cc:374] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:21:45.146191 19268 raft_consensus.cc:729] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44e218bfc262445f95be7af4a23564fa, State: Initialized, Role: FOLLOWER
I20260810 07:21:45.146607 19271 tablet_bootstrap.cc:492] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a: Bootstrap starting.
I20260810 07:21:45.146831 19268 consensus_queue.cc:260] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44e218bfc262445f95be7af4a23564fa" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37855 } } peers { permanent_uuid: "2f9797f0d02b44dc95eaebd748b0b344" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39709 } } peers { permanent_uuid: "5de4d22d341445ccb52f9093ee81623a" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46443 } }
I20260810 07:21:45.148044 19268 ts_tablet_manager.cc:1428] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260810 07:21:45.148469 18784 heartbeater.cc:503] Master 127.1.125.254:39007 was elected leader, sending a full tablet report...
I20260810 07:21:45.149446 19271 tablet_bootstrap.cc:654] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a: Neither blocks nor log segments found. Creating new log.
I20260810 07:21:45.150449 19271 log.cc:824] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:21:45.151284 19270 log.cc:824] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344: Log is configured to *not* fsync() on all Append() calls
I20260810 07:21:45.157835 19270 tablet_bootstrap.cc:492] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344: No bootstrap required, opened a new log
I20260810 07:21:45.158185 19270 ts_tablet_manager.cc:1397] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.000s
I20260810 07:21:45.158430 19271 tablet_bootstrap.cc:492] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a: No bootstrap required, opened a new log
I20260810 07:21:45.158653 19271 ts_tablet_manager.cc:1397] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260810 07:21:45.164451 19270 raft_consensus.cc:348] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44e218bfc262445f95be7af4a23564fa" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37855 } } peers { permanent_uuid: "2f9797f0d02b44dc95eaebd748b0b344" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39709 } } peers { permanent_uuid: "5de4d22d341445ccb52f9093ee81623a" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46443 } }
I20260810 07:21:45.164915 19270 raft_consensus.cc:374] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:21:45.165038 19270 raft_consensus.cc:729] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9797f0d02b44dc95eaebd748b0b344, State: Initialized, Role: FOLLOWER
I20260810 07:21:45.165697 19270 consensus_queue.cc:260] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44e218bfc262445f95be7af4a23564fa" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37855 } } peers { permanent_uuid: "2f9797f0d02b44dc95eaebd748b0b344" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39709 } } peers { permanent_uuid: "5de4d22d341445ccb52f9093ee81623a" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46443 } }
I20260810 07:21:45.167145 19270 ts_tablet_manager.cc:1428] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
I20260810 07:21:45.167631 19243 heartbeater.cc:503] Master 127.1.125.254:39007 was elected leader, sending a full tablet report...
I20260810 07:21:45.172010 19271 raft_consensus.cc:348] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44e218bfc262445f95be7af4a23564fa" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37855 } } peers { permanent_uuid: "2f9797f0d02b44dc95eaebd748b0b344" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39709 } } peers { permanent_uuid: "5de4d22d341445ccb52f9093ee81623a" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46443 } }
I20260810 07:21:45.172407 19271 raft_consensus.cc:374] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:21:45.172525 19271 raft_consensus.cc:729] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de4d22d341445ccb52f9093ee81623a, State: Initialized, Role: FOLLOWER
I20260810 07:21:45.175967 19271 consensus_queue.cc:260] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44e218bfc262445f95be7af4a23564fa" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37855 } } peers { permanent_uuid: "2f9797f0d02b44dc95eaebd748b0b344" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39709 } } peers { permanent_uuid: "5de4d22d341445ccb52f9093ee81623a" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46443 } }
I20260810 07:21:45.177528 19271 ts_tablet_manager.cc:1428] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a: Time spent starting tablet: real 0.019s user 0.008s sys 0.000s
I20260810 07:21:45.178121 19028 heartbeater.cc:503] Master 127.1.125.254:39007 was elected leader, sending a full tablet report...
W20260810 07:21:45.213439 18787 tablet.cc:2369] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 07:21:45 dist-test-slave-44d2 krb5kdc[17715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
W20260810 07:21:45.240053 19244 tablet.cc:2369] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:21:45.276284 19277 raft_consensus.cc:482] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:21:45.276599 19277 raft_consensus.cc:504] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44e218bfc262445f95be7af4a23564fa" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37855 } } peers { permanent_uuid: "2f9797f0d02b44dc95eaebd748b0b344" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39709 } } peers { permanent_uuid: "5de4d22d341445ccb52f9093ee81623a" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46443 } }
I20260810 07:21:45.278137 19277 leader_election.cc:288] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44e218bfc262445f95be7af4a23564fa (127.1.125.193:37855), 5de4d22d341445ccb52f9093ee81623a (127.1.125.194:46443)
I20260810 07:21:45.284508 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:45.278597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:46443 (local address 127.1.125.195:49153)
0810 07:21:45.279004 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:45.279020 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:45.279172 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:45.280360 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:45.280363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:45.280384 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:21:45.280526 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:45.280533 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.281633 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.281637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:45.282972 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:45.282985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.284020 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.284024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:45.284127 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.284148 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:21:45.284241 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 07:21:45.284342 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313}
I20260810 07:21:45.285861 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:45.279387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:49153 (local address 127.1.125.194:46443)
0810 07:21:45.279521 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:45.279526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:45.279539 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:21:45.279607 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:45.279610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:45.279659 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:45.279730 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:21:45.280682 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.281478 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.283129 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.283775 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.283893 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.283957 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:45.285632 (+ 1675us) server_negotiation.cc:299] Negotiation successful
0810 07:21:45.285727 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 07:21:45.286628 18923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bc0eb07cdec427d8ca52756e6a79f36" candidate_uuid: "2f9797f0d02b44dc95eaebd748b0b344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de4d22d341445ccb52f9093ee81623a" is_pre_election: true
I20260810 07:21:45.286986 18923 raft_consensus.cc:2454] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9797f0d02b44dc95eaebd748b0b344 in term 0.
I20260810 07:21:45.287832 19063 leader_election.cc:302] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9797f0d02b44dc95eaebd748b0b344, 5de4d22d341445ccb52f9093ee81623a; no voters:
I20260810 07:21:45.288179 19277 raft_consensus.cc:2781] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:21:45.288286 19277 raft_consensus.cc:482] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:21:45.288343 19277 raft_consensus.cc:3037] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:21:45.290091 19277 raft_consensus.cc:504] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44e218bfc262445f95be7af4a23564fa" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37855 } } peers { permanent_uuid: "2f9797f0d02b44dc95eaebd748b0b344" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39709 } } peers { permanent_uuid: "5de4d22d341445ccb52f9093ee81623a" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46443 } }
I20260810 07:21:45.291133 18923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bc0eb07cdec427d8ca52756e6a79f36" candidate_uuid: "2f9797f0d02b44dc95eaebd748b0b344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de4d22d341445ccb52f9093ee81623a"
I20260810 07:21:45.291291 18923 raft_consensus.cc:3037] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:21:45.291356 19277 leader_election.cc:288] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [CANDIDATE]: Term 1 election: Requested vote from peers 44e218bfc262445f95be7af4a23564fa (127.1.125.193:37855), 5de4d22d341445ccb52f9093ee81623a (127.1.125.194:46443)
I20260810 07:21:45.292243 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:45.280031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:37855 (local address 127.1.125.195:47261)
0810 07:21:45.284703 (+ 4672us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:45.284719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:21:45.284807 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:21:45.288225 (+ 3418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:21:45.288228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:21:45.288250 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:21:45.288347 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:45.288356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.289528 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.289532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:45.290756 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:21:45.290769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.291822 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.291826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:21:45.291918 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.291941 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:21:45.292013 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 07:21:45.292085 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4614,"thread_start_us":152,"threads_started":1}
I20260810 07:21:45.292659 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:45.280151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:47261 (local address 127.1.125.193:37855)
0810 07:21:45.280283 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:45.280288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:45.284901 (+ 4613us) server_negotiation.cc:407] Connection header received
0810 07:21:45.284967 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:45.284970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:45.285024 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:45.285098 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:21:45.288554 (+ 3456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.289361 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.290919 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.291555 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.291658 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.291719 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:45.292403 (+ 684us) server_negotiation.cc:299] Negotiation successful
0810 07:21:45.292519 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:21:45.292809 18923 raft_consensus.cc:2454] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9797f0d02b44dc95eaebd748b0b344 in term 1.
I20260810 07:21:45.293227 19063 leader_election.cc:302] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9797f0d02b44dc95eaebd748b0b344, 5de4d22d341445ccb52f9093ee81623a; no voters:
I20260810 07:21:45.293905 19277 raft_consensus.cc:2781] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:21:45.294006 18711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bc0eb07cdec427d8ca52756e6a79f36" candidate_uuid: "2f9797f0d02b44dc95eaebd748b0b344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44e218bfc262445f95be7af4a23564fa" is_pre_election: true
I20260810 07:21:45.294000 18709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bc0eb07cdec427d8ca52756e6a79f36" candidate_uuid: "2f9797f0d02b44dc95eaebd748b0b344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44e218bfc262445f95be7af4a23564fa"
I20260810 07:21:45.294215 19277 raft_consensus.cc:686] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9797f0d02b44dc95eaebd748b0b344, State: Running, Role: LEADER
I20260810 07:21:45.294359 18711 raft_consensus.cc:2454] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9797f0d02b44dc95eaebd748b0b344 in term 0.
I20260810 07:21:45.294603 19277 consensus_queue.cc:237] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44e218bfc262445f95be7af4a23564fa" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37855 } } peers { permanent_uuid: "2f9797f0d02b44dc95eaebd748b0b344" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39709 } } peers { permanent_uuid: "5de4d22d341445ccb52f9093ee81623a" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46443 } }
I20260810 07:21:45.298383 18444 catalog_manager.cc:5495] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9797f0d02b44dc95eaebd748b0b344 (127.1.125.195). New cstate: current_term: 1 leader_uuid: "2f9797f0d02b44dc95eaebd748b0b344" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44e218bfc262445f95be7af4a23564fa" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9797f0d02b44dc95eaebd748b0b344" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5de4d22d341445ccb52f9093ee81623a" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46443 } health_report { overall_health: UNKNOWN } } }
W20260810 07:21:45.316447 19031 tablet.cc:2369] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:21:45.329066 19264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:45.318212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.1.125.195:39709)
0810 07:21:45.318341 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 07:21:45.318346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:21:45.323677 (+ 5331us) server_negotiation.cc:407] Connection header received
0810 07:21:45.323797 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:21:45.323800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:21:45.323882 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:21:45.323994 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:21:45.324459 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.325302 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.326598 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:21:45.326943 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:21:45.327089 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:21:45.327485 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:21:45.328133 (+ 648us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:21:45.328270 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 07:21:45.328809 (+ 539us) server_negotiation.cc:299] Negotiation successful
0810 07:21:45.328884 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:21:45.337483 18709 raft_consensus.cc:3037] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:21:45.337604 18923 raft_consensus.cc:1261] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a [term 1 FOLLOWER]: Refusing update from remote peer 2f9797f0d02b44dc95eaebd748b0b344: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:21:45.338361 19277 consensus_queue.cc:1035] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5de4d22d341445ccb52f9093ee81623a" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:21:45.340370 18709 raft_consensus.cc:1261] T 5bc0eb07cdec427d8ca52756e6a79f36 P 44e218bfc262445f95be7af4a23564fa [term 1 FOLLOWER]: Refusing update from remote peer 2f9797f0d02b44dc95eaebd748b0b344: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:21:45.341197 19277 consensus_queue.cc:1035] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44e218bfc262445f95be7af4a23564fa" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 37855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:21:45.354562 19299 mvcc.cc:205] Tried to move back new op lower bound from 7316875285852020736 to 7316875285689294848. Current Snapshot: MvccSnapshot[applied={T|T < 7316875285852020736}]
I20260810 07:21:45.360822 19300 mvcc.cc:205] Tried to move back new op lower bound from 7316875285852020736 to 7316875285689294848. Current Snapshot: MvccSnapshot[applied={T|T < 7316875285852020736}]
I20260810 07:21:45.361174 19302 mvcc.cc:205] Tried to move back new op lower bound from 7316875285852020736 to 7316875285689294848. Current Snapshot: MvccSnapshot[applied={T|T < 7316875285852020736}]
I20260810 07:21:45.373502 18444 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:21:45.373704 18444 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:21:45.376904 18444 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fa3e9dc1c134fafa50870a5a40f0641: Sending DeleteTablet for 3 replicas of tablet 5bc0eb07cdec427d8ca52756e6a79f36
I20260810 07:21:45.378057 18909 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bc0eb07cdec427d8ca52756e6a79f36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:21:45 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:55822
I20260810 07:21:45.379444 18681 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bc0eb07cdec427d8ca52756e6a79f36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:21:45 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:44646
I20260810 07:21:45.379402 19134 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bc0eb07cdec427d8ca52756e6a79f36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:21:45 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:55760
I20260810 07:21:45.381052 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 18584
I20260810 07:21:45.415088 19309 tablet_replica.cc:330] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a: stopping tablet replica
I20260810 07:21:45.415347 19310 tablet_replica.cc:330] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344: stopping tablet replica
I20260810 07:21:45.419075 19310 raft_consensus.cc:2229] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:21:45.419747 19310 raft_consensus.cc:2258] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:21:45.423900 19309 raft_consensus.cc:2229] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:21:45.424430 19309 raft_consensus.cc:2258] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:21:45.431104 19310 ts_tablet_manager.cc:1905] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:21:45.431171 19309 ts_tablet_manager.cc:1905] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:21:45.434243 19309 ts_tablet_manager.cc:1918] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:21:45.434403 19310 ts_tablet_manager.cc:1918] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:21:45.434448 19309 log.cc:1196] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/5bc0eb07cdec427d8ca52756e6a79f36
I20260810 07:21:45.434659 19310 log.cc:1196] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/5bc0eb07cdec427d8ca52756e6a79f36
I20260810 07:21:45.434800 19309 ts_tablet_manager.cc:1939] T 5bc0eb07cdec427d8ca52756e6a79f36 P 5de4d22d341445ccb52f9093ee81623a: Deleting consensus metadata
I20260810 07:21:45.435117 19310 ts_tablet_manager.cc:1939] T 5bc0eb07cdec427d8ca52756e6a79f36 P 2f9797f0d02b44dc95eaebd748b0b344: Deleting consensus metadata
I20260810 07:21:45.436380 18429 catalog_manager.cc:4841] TS 2f9797f0d02b44dc95eaebd748b0b344 (127.1.125.195:39709): tablet 5bc0eb07cdec427d8ca52756e6a79f36 (table test-table [id=d90f7c0bb98f4fa38a4e65ea4c6ce580]) successfully deleted
I20260810 07:21:45.436384 18428 catalog_manager.cc:4841] TS 5de4d22d341445ccb52f9093ee81623a (127.1.125.194:46443): tablet 5bc0eb07cdec427d8ca52756e6a79f36 (table test-table [id=d90f7c0bb98f4fa38a4e65ea4c6ce580]) successfully deleted
W20260810 07:21:45.436944 18429 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:37855 (error 108)
I20260810 07:21:45.438616 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 18798
I20260810 07:21:45.438650 19266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:21:45.438327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:37855 (local address 127.0.0.1:44650)
0810 07:21:45.438465 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 07:21:45.438561 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:37855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 07:21:45.441108 18429 catalog_manager.cc:4584] TS 44e218bfc262445f95be7af4a23564fa (127.1.125.193:37855): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5bc0eb07cdec427d8ca52756e6a79f36: Network error: Client connection negotiation failed: client connection to 127.1.125.193:37855: connect: Connection refused (error 111)
I20260810 07:21:45.450537 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 19037
I20260810 07:21:45.493826 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 18402
I20260810 07:21:45.506763 1527 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 07:21:45.853168 1527 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 07:21:45.853300 1527 mini_ranger.cc:66] Stopping Ranger...
I20260810 07:21:46.218703 1527 mini_ranger.cc:68] Stopped Ranger
2026-08-10 07:21:46.218 UTC [17764] LOG: received smart shutdown request
2026-08-10 07:21:46.223 UTC [17764] LOG: background worker "logical replication launcher" (PID 17778) exited with exit code 1
2026-08-10 07:21:46.223 UTC [17772] LOG: shutting down
2026-08-10 07:21:46.398 UTC [17764] LOG: database system is shut down
2026-08-10T07:21:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51288 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:21:46 dist-test-slave-44d2 krb5kdc[19331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:21:46 dist-test-slave-44d2 krb5kdc[19331](info): set up 2 sockets
Aug 10 07:21:46 dist-test-slave-44d2 krb5kdc[19331](info): commencing operation
krb5kdc: starting...
W20260810 07:21:47.779328 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:21:47 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:21:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:21:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/rangeradmin_127.1.125.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/rangeradmin_127.1.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/rangerlookup_127.1.125.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/rangerlookup_127.1.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/HTTP_127.1.125.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/HTTP_127.1.125.212@KRBTEST.COM.keytab.
I20260810 07:21:47.966985 1527 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwKbuEn/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/postgres -l logfile start
2026-08-10 07:21:50.597 UTC [19376] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 07:21:50.597 UTC [19376] LOG: listening on IPv4 address "127.1.125.212", port 45989
2026-08-10 07:21:50.601 UTC [19376] LOG: listening on Unix socket "/tmp/.s.PGSQL.45989"
2026-08-10 07:21:50.623 UTC [19379] LOG: database system was shut down at 2026-08-10 07:21:49 UTC
2026-08-10 07:21:50.627 UTC [19376] LOG: database system is ready to accept connections
I20260810 07:21:51.834568 1527 mini_postgres.cc:96] Postgres bound to 45989
2026-08-10 07:21:51.842 UTC [19391] FATAL: database "slave" does not exist
127.1.125.212:45989 - accepting connections
I20260810 07:21:51.843207 1527 mini_ranger.cc:161] Starting Ranger...
I20260810 07:21:51.860378 1527 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 07:21:52.336143 1527 mini_ranger.cc:87] Created ranger Postgres database
I20260810 07:21:52.336242 1527 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-admin
2026-08-10 07:21:52,619 [I] DB FLAVOR :POSTGRES
2026-08-10 07:21:52,619 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:21:52,619 [I] Checking connection..
2026-08-10 07:21:52,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 07:21:52,887 [I] Checking connection passed.
2026-08-10 07:21:52,888 [I] --------- Verifying version history table ---------
2026-08-10 07:21:52,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:21:53,138 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:21:53,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:21:53,400 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:21:53,400 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 07:21:53,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 07:21:53.667 UTC [19479] WARNING: there is no transaction in progress
2026-08-10 07:21:53,674 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 07:21:53,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:21:53,918 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 07:21:53,918 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:21:53,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 07:21:54,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 07:21:54,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-44d2.c.gcp-upstream.internal','N') ;"
2026-08-10 07:21:54,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:21:54,953 [I] Table x_portal_user does not exist in database ranger
2026-08-10 07:21:54,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:21:55,232 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 07:21:55,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:21:55,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:21:55,767 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 07:21:55,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 07:21:56.731 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.747 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.763 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.778 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.797 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.815 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.872 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.879 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.894 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.905 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.915 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.929 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.938 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.945 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.952 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:56.959 UTC [19631] WARNING: there is no transaction in progress
2026-08-10 07:21:57,609 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 07:21:57,609 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 07:21:57,872 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 07:21:57,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:21:58,136 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 07:21:58,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:21:58,414 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 07:21:58,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:21:58,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:21:58,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-44d2.c.gcp-upstream.internal';"
2026-08-10 07:21:59,200 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 07:21:59,201 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 07:21:59,480 [I] DB_PATCHES have already been applied
I20260810 07:21:59.486840 1527 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwKbuEn/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 07:21:59.486912 1527 mini_ranger.cc:193] Using host: 127.1.125.212
I20260810 07:21:59.491604 1527 mini_ranger.cc:227] Ranger admin URL: http://127.1.125.212:54481
Aug 10, 2026 7:21:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:21:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:21:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 7:21:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 7:21:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 7:22:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 7:22:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 07:22:00.899866 1527 mini_ranger.cc:160] Time spent starting Ranger: real 9.057s user 0.000s sys 0.013s
Aug 10, 2026 7:22:20 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 7:22:20 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 10, 2026 7:22:20 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 7:22:21 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 7:22:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 7:22:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 7:22:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 7:22:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 7:22:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 7:22:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 7:22:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 7:22:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 7:22:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 7:22:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 7:22:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 7:22:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 7:22:21 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 7:22:22 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 07:22:22.770776 1527 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/rangerkms_127.1.125.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/rangerkms_127.1.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.125.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.125.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.125.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/HTTP_127.1.125.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.125.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/HTTP_127.1.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 07:22:22 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346542, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 07:22:22.880411 1527 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 07:22:22.892633 1527 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 07:22:23.763157 1527 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 07:22:23.763285 1527 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-kms
I20260810 07:22:23.763314 1527 mini_ranger_kms.cc:227] Using postgres at 127.1.125.212:45989
2026-08-10 07:22:24,061 [I] DB FLAVOR :POSTGRES
2026-08-10 07:22:24,061 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:22:24,062 [I] Checking connection
2026-08-10 07:22:24,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 07:22:24,316 [I] connection success
2026-08-10 07:22:24,316 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 07:22:24,316 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 07:22:24,316 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 07:22:24,551 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 07:22:24,552 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:22:24,552 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 07:22:24,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.125.212:45989/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 07:22:24,833 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 07:22:25.355518 1527 mini_ranger_kms.cc:310] Created kms service
I20260810 07:22:25.588626 1527 mini_ranger_kms.cc:326] Created kudu user
I20260810 07:22:25.636147 1527 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 07:22:25 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346542, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:22:26.020715 1527 mini_ranger_kms.cc:384] Added ranger policy
I20260810 07:22:26.020893 1527 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwKbuEn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwKbuEn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 07:22:26.020922 1527 mini_ranger_kms.cc:240] Using host: 127.1.125.212
I20260810 07:22:26.027031 1527 mini_ranger_kms.cc:276] Ranger KMS PID: 19921
I20260810 07:22:26.027140 1527 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.125.212:38223
Aug 10, 2026 7:22:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:22:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:22:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 7:22:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 7:22:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 7:22:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 07:22:28 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346548, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.125.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:28.847043 1527 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.967s user 0.016s sys 0.252s
I20260810 07:22:28.847203 1527 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 07:22:28.847290 1527 mini_ranger_kms.cc:401] 127.1.125.212:38223/kms/v1/keys
Aug 10 07:22:29 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346548, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.125.212@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
Aug 10 07:22:29 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346548, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.125.212@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
Aug 10 07:22:31 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:22:31.847568 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38615
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:38615
--ranger_config_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.125.212:38223/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:31.983736 19990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:31.984026 19990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:31.984066 19990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:31.988866 19990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:31.988933 19990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:31.988955 19990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:31.988971 19990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:31.989056 19990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:31.995270 19990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.125.212:38223/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:38615
--ranger_config_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:31.996455 19990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:31.997648 19990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:32.007191 20001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:32.007259 19996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:32.007164 19998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:32.007354 19990 server_base.cc:1034] running on GCE node
I20260810 07:22:32.008140 19990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:32.009351 19990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:32.010555 19990 hybrid_clock.cc:648] HybridClock initialized: now 1786346552010536 us; error 33 us; skew 500 ppm
Aug 10 07:22:32 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346552, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:32.013600 19990 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:22:32.014437 19990 webserver.cc:460] Webserver started at http://127.1.125.254:37317/ using document root <none> and password file <none>
I20260810 07:22:32.014974 19990 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:32.015029 19990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:32.015208 19990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:22:32 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346552, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:22:32.063504 19990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "e72daf618e804bab849652a057827fb4"
format_stamp: "Formatted at 2026-08-10 07:22:32 on dist-test-slave-44d2"
server_key: "0e184cbb4b03d99ac50cd5bdbda363a6"
server_key_iv: "d1f19bc56bff41098fedf0cc36073247"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:32.064406 19990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "e72daf618e804bab849652a057827fb4"
format_stamp: "Formatted at 2026-08-10 07:22:32 on dist-test-slave-44d2"
server_key: "0e184cbb4b03d99ac50cd5bdbda363a6"
server_key_iv: "d1f19bc56bff41098fedf0cc36073247"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:32.069597 19990 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 07:22:32.087353 20009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:32.088901 19990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:22:32.089054 19990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "e72daf618e804bab849652a057827fb4"
format_stamp: "Formatted at 2026-08-10 07:22:32 on dist-test-slave-44d2"
server_key: "0e184cbb4b03d99ac50cd5bdbda363a6"
server_key_iv: "d1f19bc56bff41098fedf0cc36073247"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:32.089195 19990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:32.109040 19990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:32.111598 19990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:32.111816 19990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:32.119758 19990 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:38615
I20260810 07:22:32.119764 20093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:38615 every 8 connection(s)
I20260810 07:22:32.120846 19990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:32.123672 20095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:32.126634 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 19990
I20260810 07:22:32.126782 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:22:32.127107 1527 external_mini_cluster.cc:1428] Setting key 243266916129f3b0ef26ff979789498c
I20260810 07:22:32.129838 20095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4: Bootstrap starting.
I20260810 07:22:32.132623 20095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:32.133462 20095 log.cc:824] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:32.135527 20095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4: No bootstrap required, opened a new log
Aug 10 07:22:32 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:32.140110 20095 raft_consensus.cc:348] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72daf618e804bab849652a057827fb4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38615 } }
I20260810 07:22:32.140377 20095 raft_consensus.cc:374] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:32.140429 20095 raft_consensus.cc:729] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72daf618e804bab849652a057827fb4, State: Initialized, Role: FOLLOWER
I20260810 07:22:32.140919 20095 consensus_queue.cc:260] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72daf618e804bab849652a057827fb4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38615 } }
I20260810 07:22:32.141115 20095 raft_consensus.cc:388] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:32.141225 20095 raft_consensus.cc:482] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:32.141341 20095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:32.142390 20095 raft_consensus.cc:504] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72daf618e804bab849652a057827fb4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38615 } }
I20260810 07:22:32.142522 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:32.128643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55988 (local address 127.1.125.254:38615)
0810 07:22:32.129064 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:32.129084 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:32.129129 (+ 45us) server_negotiation.cc:407] Connection header received
0810 07:22:32.129942 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:32.129966 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:32.130361 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:32.130726 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:32.131423 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:32.132947 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:32.133915 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:32.134527 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:32.134679 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:32.137299 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:32.137308 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:32.137325 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:32.137356 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:32.139220 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:32.139645 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:32.139651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:32.139656 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:32.139791 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:32.140056 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:32.140059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:32.140060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:32.140593 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:32.140794 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:32.141113 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 07:22:32.141572 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":112,"threads_started":1}
I20260810 07:22:32.142751 20095 leader_election.cc:302] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e72daf618e804bab849652a057827fb4; no voters:
I20260810 07:22:32.143092 20095 leader_election.cc:288] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:22:32.143213 20106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:32.143438 20106 raft_consensus.cc:686] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 1 LEADER]: Becoming Leader. State: Replica: e72daf618e804bab849652a057827fb4, State: Running, Role: LEADER
I20260810 07:22:32.144268 20095 sys_catalog.cc:564] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:32.144770 20106 consensus_queue.cc:237] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72daf618e804bab849652a057827fb4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38615 } }
I20260810 07:22:32.146670 20095 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 07:22:32.146740 20109 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e72daf618e804bab849652a057827fb4. Latest consensus state: current_term: 1 leader_uuid: "e72daf618e804bab849652a057827fb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72daf618e804bab849652a057827fb4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38615 } } }
I20260810 07:22:32.146930 20109 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:32.146740 20107 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e72daf618e804bab849652a057827fb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72daf618e804bab849652a057827fb4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38615 } } }
I20260810 07:22:32.147293 20107 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:32.147332 20125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:32.154484 20125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:32.160993 20125 catalog_manager.cc:1347] Generated new cluster ID: 4e8a6c0d5b4d4fe2add8fd8bdce8b02a
I20260810 07:22:32.161056 20125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:22:32.187408 20125 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:22:32.188411 20125 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:32.195361 20125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4: Generated new TSK 0
I20260810 07:22:32.196077 20125 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:22:32.924419 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.125.212:38223/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38615
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:33.070273 20165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:33.070654 20165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:33.070720 20165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:33.076788 20165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:33.076953 20165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:33.077054 20165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:22:33.084120 20165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.125.212:38223/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:33.085444 20165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:33.086823 20165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:33.098140 20175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:33.098628 20178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:33.099620 20165 server_base.cc:1034] running on GCE node
W20260810 07:22:33.100215 20173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:33.100770 20165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:33.101519 20165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:33.102802 20165 hybrid_clock.cc:648] HybridClock initialized: now 1786346553102736 us; error 82 us; skew 500 ppm
Aug 10 07:22:33 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:33.106671 20165 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:22:33.107733 20165 webserver.cc:460] Webserver started at http://127.1.125.193:39039/ using document root <none> and password file <none>
I20260810 07:22:33.108533 20165 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:33.108613 20165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:33.108841 20165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:22:33 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:22:33.126104 20165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "f74abe29c92a4888afd4c04adcc5a586"
format_stamp: "Formatted at 2026-08-10 07:22:33 on dist-test-slave-44d2"
server_key: "8c838d54be0c0d05978d862058cf8b83"
server_key_iv: "3a85d9aeb6b46dcad35a0c3d3789d852"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:33.127279 20165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "f74abe29c92a4888afd4c04adcc5a586"
format_stamp: "Formatted at 2026-08-10 07:22:33 on dist-test-slave-44d2"
server_key: "8c838d54be0c0d05978d862058cf8b83"
server_key_iv: "3a85d9aeb6b46dcad35a0c3d3789d852"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:33.132472 20165 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 07:22:33.161823 20192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:33.163736 20165 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.003s sys 0.000s
I20260810 07:22:33.163904 20165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "f74abe29c92a4888afd4c04adcc5a586"
format_stamp: "Formatted at 2026-08-10 07:22:33 on dist-test-slave-44d2"
server_key: "8c838d54be0c0d05978d862058cf8b83"
server_key_iv: "3a85d9aeb6b46dcad35a0c3d3789d852"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:33.164060 20165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:33.191812 20165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:33.196156 20165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:33.196384 20165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:33.197183 20165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:33.198376 20165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:33.198460 20165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:33.198594 20165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:33.198663 20165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:33.211483 20165 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:44077
I20260810 07:22:33.211666 20355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:44077 every 8 connection(s)
I20260810 07:22:33.212800 20165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:22:33.218212 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 20165
I20260810 07:22:33.218343 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:22:33.218704 1527 external_mini_cluster.cc:1428] Setting key a6a9a77e9426272fbda7ac0a72e5a1a9
Aug 10 07:22:33 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:22:33.232954 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:33.214927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:59945 (local address 127.1.125.254:38615)
0810 07:22:33.215273 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:33.215280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:33.216201 (+ 921us) server_negotiation.cc:407] Connection header received
0810 07:22:33.217540 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:33.217544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:33.217627 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:33.217746 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:33.219313 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:33.220183 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:33.222429 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:33.222867 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:33.222972 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:33.226613 (+ 3641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:33.226620 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:33.226627 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:33.226663 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:33.229483 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:33.230251 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:33.230257 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:33.230260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:33.230387 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:33.231246 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:33.231251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:33.231253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:33.231552 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:33.231661 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:33.232533 (+ 872us) server_negotiation.cc:299] Negotiation successful
0810 07:22:33.232712 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":260,"thread_start_us":100,"threads_started":1}
I20260810 07:22:33.234673 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:33.215417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38615 (local address 127.1.125.193:59945)
0810 07:22:33.216020 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:33.216083 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:33.217262 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:33.217964 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:33.217977 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:33.218391 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:33.219102 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:33.219125 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:33.220369 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:33.220375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:33.222241 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:33.222259 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:33.223235 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:33.223242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:33.223504 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:33.224281 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:33.224312 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:33.226383 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:33.229656 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:33.229665 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:33.229687 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:33.230098 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:33.230526 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:33.230529 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:33.230532 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:33.231099 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:33.231746 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:33.231757 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:33.232130 (+ 373us) client_negotiation.cc:769] Sending connection context
0810 07:22:33.233215 (+ 1085us) client_negotiation.cc:240] Negotiation successful
0810 07:22:33.233626 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":180,"threads_started":1}
I20260810 07:22:33.236397 20356 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
I20260810 07:22:33.236717 20356 heartbeater.cc:464] Registering TS with master...
Aug 10 07:22:33 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:33.237392 20356 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
I20260810 07:22:33.240252 20035 ts_manager.cc:194] Registered new tserver with Master: f74abe29c92a4888afd4c04adcc5a586 (127.1.125.193:44077)
I20260810 07:22:33.241719 20035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:59945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:22:33.259717 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.125.212:38223/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38615
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:33.392700 20371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:33.393186 20371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:33.393350 20371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:33.398945 20371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:33.399034 20371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:33.399161 20371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:22:33.405308 20371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.125.212:38223/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:33.406586 20371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:33.407887 20371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:33.419287 20381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:33.419356 20379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:33.419787 20386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:33.421212 20371 server_base.cc:1034] running on GCE node
I20260810 07:22:33.421636 20371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:33.422402 20371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:33.423635 20371 hybrid_clock.cc:648] HybridClock initialized: now 1786346553423595 us; error 50 us; skew 500 ppm
Aug 10 07:22:33 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:33.427469 20371 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:22:33.428650 20371 webserver.cc:460] Webserver started at http://127.1.125.194:44687/ using document root <none> and password file <none>
I20260810 07:22:33.429456 20371 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:33.429555 20371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:33.429781 20371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:22:33 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:22:33.445916 20371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "aaf73521d83c4e60835c72b7ff03e392"
format_stamp: "Formatted at 2026-08-10 07:22:33 on dist-test-slave-44d2"
server_key: "476b5a8f528e8ad79f4f403d50ee4c24"
server_key_iv: "214c4aafdc4adf1aaf5da9039b5efb43"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:33.446714 20371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "aaf73521d83c4e60835c72b7ff03e392"
format_stamp: "Formatted at 2026-08-10 07:22:33 on dist-test-slave-44d2"
server_key: "476b5a8f528e8ad79f4f403d50ee4c24"
server_key_iv: "214c4aafdc4adf1aaf5da9039b5efb43"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:33.453665 20371 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260810 07:22:33.486281 20402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:33.488073 20371 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.002s
I20260810 07:22:33.488209 20371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "aaf73521d83c4e60835c72b7ff03e392"
format_stamp: "Formatted at 2026-08-10 07:22:33 on dist-test-slave-44d2"
server_key: "476b5a8f528e8ad79f4f403d50ee4c24"
server_key_iv: "214c4aafdc4adf1aaf5da9039b5efb43"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:33.488349 20371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:33.515786 20371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:33.519531 20371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:33.519752 20371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:33.520437 20371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:33.521468 20371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:33.521545 20371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:33.521651 20371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:33.521698 20371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:33.533736 20371 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:34865
I20260810 07:22:33.533771 20570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:34865 every 8 connection(s)
I20260810 07:22:33.534874 20371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:22:33.543723 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 20371
I20260810 07:22:33.543874 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:22:33.544263 1527 external_mini_cluster.cc:1428] Setting key 6d4170a578a4a0fdb5656a177ac4660e
Aug 10 07:22:33 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:33.551880 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:33.537017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:58399 (local address 127.1.125.254:38615)
0810 07:22:33.537154 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:33.537158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:33.538138 (+ 980us) server_negotiation.cc:407] Connection header received
0810 07:22:33.539165 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:33.539169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:33.539233 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:33.539338 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:33.540459 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:33.541161 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:33.542332 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:33.542582 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:33.542698 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:33.546862 (+ 4164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:33.546868 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:33.546874 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:33.546922 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:33.548996 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:33.549715 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:33.549721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:33.549723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:33.549814 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:33.550289 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:33.550294 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:33.550296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:33.550547 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:33.550629 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:33.551463 (+ 834us) server_negotiation.cc:299] Negotiation successful
0810 07:22:33.551625 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:22:33.553835 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:33.537462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38615 (local address 127.1.125.194:58399)
0810 07:22:33.537974 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:33.538028 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:33.538940 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:33.539526 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:33.539534 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:33.539813 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:33.540298 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:33.540311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:33.541294 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:33.541298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:33.542188 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:33.542198 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:33.542740 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:33.542745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:33.542927 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:33.544829 (+ 1902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:33.544852 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:33.546657 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:33.549150 (+ 2493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:33.549159 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:33.549180 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:33.549585 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:33.549965 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:33.549969 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:33.549973 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:33.550171 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:33.550724 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:33.550734 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:33.551125 (+ 391us) client_negotiation.cc:769] Sending connection context
0810 07:22:33.552149 (+ 1024us) client_negotiation.cc:240] Negotiation successful
0810 07:22:33.552570 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":151,"threads_started":1}
I20260810 07:22:33.555420 20571 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38615
I20260810 07:22:33.555673 20571 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:33.556259 20571 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
I20260810 07:22:33.564901 20035 ts_manager.cc:194] Registered new tserver with Master: aaf73521d83c4e60835c72b7ff03e392 (127.1.125.194:34865)
I20260810 07:22:33.565691 20035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:58399
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:22:33.612236 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.125.212:38223/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38615
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:22:33 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
W20260810 07:22:33.878772 20584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:33.879302 20584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:33.879410 20584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:33.885545 20584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:33.885707 20584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:33.885820 20584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:22:33.895424 20584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.125.212:38223/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:33.896939 20584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:33.898600 20584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:33.912272 20590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:33.912288 20589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:33.912853 20592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:33.913452 20584 server_base.cc:1034] running on GCE node
I20260810 07:22:33.913919 20584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:33.914680 20584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:33.915920 20584 hybrid_clock.cc:648] HybridClock initialized: now 1786346553915848 us; error 85 us; skew 500 ppm
Aug 10 07:22:33 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:33.919786 20584 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:22:33.921097 20584 webserver.cc:460] Webserver started at http://127.1.125.195:40213/ using document root <none> and password file <none>
I20260810 07:22:33.921914 20584 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:33.922073 20584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:33.922408 20584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:22:33 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:22:33.948588 20584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "114dbfae2aec427cb5173f9297fa9644"
format_stamp: "Formatted at 2026-08-10 07:22:33 on dist-test-slave-44d2"
server_key: "25cb843bc62fbc4ebfc04db796f1b039"
server_key_iv: "6ed45f099a14bbb60bece6c2b8e01932"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:33.949743 20584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "114dbfae2aec427cb5173f9297fa9644"
format_stamp: "Formatted at 2026-08-10 07:22:33 on dist-test-slave-44d2"
server_key: "25cb843bc62fbc4ebfc04db796f1b039"
server_key_iv: "6ed45f099a14bbb60bece6c2b8e01932"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:33.955585 20584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 07:22:33.984970 20601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:33.986804 20584 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260810 07:22:33.986953 20584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "114dbfae2aec427cb5173f9297fa9644"
format_stamp: "Formatted at 2026-08-10 07:22:33 on dist-test-slave-44d2"
server_key: "25cb843bc62fbc4ebfc04db796f1b039"
server_key_iv: "6ed45f099a14bbb60bece6c2b8e01932"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:33.987100 20584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:34.018683 20584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:34.022542 20584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:34.022820 20584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:34.023617 20584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:34.024780 20584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:34.024907 20584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:34.025019 20584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:34.025087 20584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:34.038185 20584 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:37635
I20260810 07:22:34.039479 20584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:22:34.047339 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 20584
I20260810 07:22:34.047477 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:22:34.047847 1527 external_mini_cluster.cc:1428] Setting key 0fe1ae11ec05966495ea679dbcdb9a13
I20260810 07:22:34.056511 20780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:37635 every 8 connection(s)
Aug 10 07:22:34 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:34.090021 20787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:34.054830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38615 (local address 127.1.125.195:33999)
0810 07:22:34.055467 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:34.055536 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:34.059176 (+ 3640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:34.068844 (+ 9668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:34.068856 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:34.069368 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:34.070344 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:34.070369 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:34.071620 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:34.071623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:34.074799 (+ 3176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:34.074812 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:34.075626 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:34.075630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:34.075837 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:34.076608 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:34.076632 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:34.078563 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:34.080839 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:34.080847 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:34.080865 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:34.081237 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:34.081593 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:34.081596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:34.081598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:34.081797 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:34.087798 (+ 6001us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:34.087806 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:34.088203 (+ 397us) client_negotiation.cc:769] Sending connection context
0810 07:22:34.088437 (+ 234us) client_negotiation.cc:240] Negotiation successful
0810 07:22:34.088846 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":159,"threads_started":1}
I20260810 07:22:34.090827 20786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:34.054221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:33999 (local address 127.1.125.254:38615)
0810 07:22:34.067034 (+ 12813us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:34.067040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:34.067057 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:22:34.067103 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:34.067106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:34.067163 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:34.067303 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:34.070543 (+ 3240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:34.071450 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:34.074970 (+ 3520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:34.075328 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:34.075435 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:34.078773 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:34.078777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:34.078780 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:34.078810 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:34.080702 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:34.081373 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:34.081377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:34.081379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:34.081480 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:34.081933 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:34.081936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:34.081938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:34.087625 (+ 5687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:34.087734 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:34.090509 (+ 2775us) server_negotiation.cc:299] Negotiation successful
0810 07:22:34.090655 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12716,"thread_start_us":87,"threads_started":1}
I20260810 07:22:34.092579 20782 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38615
I20260810 07:22:34.092911 20782 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:34.093555 20782 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
I20260810 07:22:34.094991 20035 ts_manager.cc:194] Registered new tserver with Master: 114dbfae2aec427cb5173f9297fa9644 (127.1.125.195:37635)
I20260810 07:22:34.095741 20035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:33999
I20260810 07:22:34.099536 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:22:34.100356 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 20165
I20260810 07:22:34.119855 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 20371
I20260810 07:22:34.134567 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 20584
Aug 10 07:22:34 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:22:34.155359 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 19990
I20260810 07:22:34.175627 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38615
--webserver_interface=127.1.125.254
--webserver_port=37317
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:38615
--ranger_config_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.125.212:38223/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.125.212:38223/kms with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:34.293612 20792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:34.293874 20792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:34.293919 20792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:34.297904 20792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:34.297950 20792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:34.297968 20792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:34.297984 20792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:34.297999 20792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:34.303411 20792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.125.212:38223/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:38615
--ranger_config_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=37317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:34.304385 20792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:34.305418 20792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:34.314090 20800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:34.314338 20792 server_base.cc:1034] running on GCE node
W20260810 07:22:34.314081 20802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:34.314085 20799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:34.314972 20792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:34.316061 20792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:34.317222 20792 hybrid_clock.cc:648] HybridClock initialized: now 1786346554317205 us; error 31 us; skew 500 ppm
Aug 10 07:22:34 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:34.320086 20792 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:22:34.320971 20792 webserver.cc:460] Webserver started at http://127.1.125.254:37317/ using document root <none> and password file <none>
I20260810 07:22:34.321478 20792 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:34.321552 20792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:22:34 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:22:34.407174 20792 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 07:22:34.409499 20817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:34.410859 20792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:22:34.411021 20792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "e72daf618e804bab849652a057827fb4"
format_stamp: "Formatted at 2026-08-10 07:22:32 on dist-test-slave-44d2"
server_key: "0e184cbb4b03d99ac50cd5bdbda363a6"
server_key_iv: "d1f19bc56bff41098fedf0cc36073247"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:34.411616 20792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:34.434170 20792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:34.436894 20792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:34.437093 20792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:34.444666 20792 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:38615
I20260810 07:22:34.444694 20906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:38615 every 8 connection(s)
I20260810 07:22:34.445739 20792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:34.450124 20907 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:22:34.451758 20907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4: Bootstrap starting.
I20260810 07:22:34.454618 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 20792
I20260810 07:22:34.455082 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.125.212:38223/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:44077
--local_ip_for_outbound_sockets=127.1.125.193
--tserver_master_addrs=127.1.125.254:38615
--webserver_port=39039
--webserver_interface=127.1.125.193
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.125.212:38223/kms with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:34.469619 20907 log.cc:824] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:34.477041 20907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:34.477483 20907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4: Bootstrap complete.
I20260810 07:22:34.482009 20907 raft_consensus.cc:348] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72daf618e804bab849652a057827fb4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38615 } }
I20260810 07:22:34.482439 20907 raft_consensus.cc:729] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72daf618e804bab849652a057827fb4, State: Initialized, Role: FOLLOWER
I20260810 07:22:34.482996 20907 consensus_queue.cc:260] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72daf618e804bab849652a057827fb4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38615 } }
I20260810 07:22:34.483150 20907 raft_consensus.cc:388] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:34.483242 20907 raft_consensus.cc:482] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:34.483363 20907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:22:34.485527 20907 raft_consensus.cc:504] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72daf618e804bab849652a057827fb4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38615 } }
I20260810 07:22:34.485896 20907 leader_election.cc:302] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e72daf618e804bab849652a057827fb4; no voters:
I20260810 07:22:34.486284 20907 leader_election.cc:288] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:22:34.486428 20914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:22:34.486676 20914 raft_consensus.cc:686] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [term 2 LEADER]: Becoming Leader. State: Replica: e72daf618e804bab849652a057827fb4, State: Running, Role: LEADER
I20260810 07:22:34.487093 20914 consensus_queue.cc:237] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72daf618e804bab849652a057827fb4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38615 } }
I20260810 07:22:34.487543 20907 sys_catalog.cc:564] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:34.488636 20915 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e72daf618e804bab849652a057827fb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72daf618e804bab849652a057827fb4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38615 } } }
I20260810 07:22:34.488770 20915 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:34.489140 20916 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e72daf618e804bab849652a057827fb4. Latest consensus state: current_term: 2 leader_uuid: "e72daf618e804bab849652a057827fb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72daf618e804bab849652a057827fb4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38615 } } }
I20260810 07:22:34.489310 20916 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:34.489665 20926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:34.490038 20907 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 07:22:34.497066 20926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:34.498135 20926 catalog_manager.cc:1259] Loaded cluster ID: 4e8a6c0d5b4d4fe2add8fd8bdce8b02a
I20260810 07:22:34.498245 20926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:22:34.501711 20926 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:34.502511 20926 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4: Loaded TSK: 0
I20260810 07:22:34.503868 20926 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:22:34.583283 20912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:34.583633 20912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:34.583703 20912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:34.590273 20912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:34.590422 20912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:34.590617 20912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:22:34.598996 20912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.125.212:38223/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:44077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=39039
--tserver_master_addrs=127.1.125.254:38615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:34.600596 20912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:34.602098 20912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:34.615149 20963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:34.615260 20968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:34.615124 20962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:34.616396 20912 server_base.cc:1034] running on GCE node
I20260810 07:22:34.616902 20912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:34.617779 20912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:34.619002 20912 hybrid_clock.cc:648] HybridClock initialized: now 1786346554618978 us; error 37 us; skew 500 ppm
Aug 10 07:22:34 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:34.622918 20912 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:22:34.624094 20912 webserver.cc:460] Webserver started at http://127.1.125.193:39039/ using document root <none> and password file <none>
I20260810 07:22:34.624917 20912 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:34.625104 20912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:22:34 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:22:34.646955 20912 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 07:22:34.650512 20979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:34.652184 20912 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 07:22:34.652330 20912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "f74abe29c92a4888afd4c04adcc5a586"
format_stamp: "Formatted at 2026-08-10 07:22:33 on dist-test-slave-44d2"
server_key: "8c838d54be0c0d05978d862058cf8b83"
server_key_iv: "3a85d9aeb6b46dcad35a0c3d3789d852"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:34.653138 20912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:34.677762 20912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:34.681843 20912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:34.682173 20912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:34.682945 20912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:34.684185 20912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:34.684319 20912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:34.684520 20912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:34.684652 20912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:34.695705 20912 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:44077
I20260810 07:22:34.696237 21157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:44077 every 8 connection(s)
I20260810 07:22:34.697176 20912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:22:34.698657 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 20912
I20260810 07:22:34.699174 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.125.212:38223/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:34865
--local_ip_for_outbound_sockets=127.1.125.194
--tserver_master_addrs=127.1.125.254:38615
--webserver_port=44687
--webserver_interface=127.1.125.194
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.125.212:38223/kms with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:22:34 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:34.722065 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:34.701648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38615 (local address 127.1.125.193:57993)
0810 07:22:34.703083 (+ 1435us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:34.703168 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:34.704350 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:34.705918 (+ 1568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:34.705932 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:34.706360 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:34.707124 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:34.707151 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:34.709343 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:34.709348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:34.710961 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:34.710976 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:34.712505 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:34.712514 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:34.712795 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:34.713634 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:34.713676 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:34.715713 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:34.718386 (+ 2673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:34.718392 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:34.718406 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:34.718770 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:34.719178 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:34.719181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:34.719183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:34.719354 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:34.720136 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:34.720143 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:34.720429 (+ 286us) client_negotiation.cc:769] Sending connection context
0810 07:22:34.720666 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 07:22:34.721000 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1201,"thread_start_us":158,"threads_started":1}
I20260810 07:22:34.723961 21162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:34.701660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:57993 (local address 127.1.125.254:38615)
0810 07:22:34.702896 (+ 1236us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:34.702921 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:34.703604 (+ 683us) server_negotiation.cc:407] Connection header received
0810 07:22:34.704886 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:34.704903 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:34.705347 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:34.705760 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:34.707365 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:34.709155 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:34.711132 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:34.711998 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:34.712198 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:34.715946 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:34.715955 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:34.715972 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:34.716007 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:34.718245 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:34.718913 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:34.718920 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:34.718925 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:34.719067 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:34.719459 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:34.719462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:34.719464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:34.719948 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:34.720076 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:34.722638 (+ 2562us) server_negotiation.cc:299] Negotiation successful
0810 07:22:34.722997 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1040,"thread_start_us":135,"threads_started":1}
I20260810 07:22:34.726526 21158 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38615
I20260810 07:22:34.726806 21158 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:34.727396 21158 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
W20260810 07:22:34.729882 21158 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:38615 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:22:34.730010 21158 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:34.730208 21158 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
I20260810 07:22:34.731208 21158 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:34.731374 21158 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
W20260810 07:22:34.732188 21158 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:22:34.849521 21161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:34.849802 21161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:34.849874 21161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:34.854631 21161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:34.854710 21161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:34.854797 21161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:22:34.860790 21161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.125.212:38223/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:34865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=44687
--tserver_master_addrs=127.1.125.254:38615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:34.862149 21161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:34.863502 21161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:34.877835 21177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:34.877890 21174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:34.878293 21161 server_base.cc:1034] running on GCE node
W20260810 07:22:34.878944 21175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:34.879523 21161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:34.880215 21161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:34.881403 21161 hybrid_clock.cc:648] HybridClock initialized: now 1786346554881369 us; error 49 us; skew 500 ppm
Aug 10 07:22:34 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:34.885011 21161 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:22:34.886196 21161 webserver.cc:460] Webserver started at http://127.1.125.194:44687/ using document root <none> and password file <none>
I20260810 07:22:34.886963 21161 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:34.887089 21161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:22:34 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:22:34.910488 21161 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 07:22:34.913918 21189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:34.915693 21161 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 07:22:34.915839 21161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "aaf73521d83c4e60835c72b7ff03e392"
format_stamp: "Formatted at 2026-08-10 07:22:33 on dist-test-slave-44d2"
server_key: "476b5a8f528e8ad79f4f403d50ee4c24"
server_key_iv: "214c4aafdc4adf1aaf5da9039b5efb43"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:34.916679 21161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:34.947083 21161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:34.951206 21161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:34.951514 21161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:34.952399 21161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:34.953696 21161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:34.953805 21161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:34.953904 21161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:34.953974 21161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:34.968676 21161 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:34865
I20260810 07:22:34.968758 21360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:34865 every 8 connection(s)
I20260810 07:22:34.970228 21161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:22:34.970652 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 21161
I20260810 07:22:34.971220 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.125.212:38223/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:37635
--local_ip_for_outbound_sockets=127.1.125.195
--tserver_master_addrs=127.1.125.254:38615
--webserver_port=40213
--webserver_interface=127.1.125.195
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.125.212:38223/kms with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:22:34 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:35.003331 21162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:34.974279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:48683 (local address 127.1.125.254:38615)
0810 07:22:34.974418 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:34.974423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:34.988435 (+ 14012us) server_negotiation.cc:407] Connection header received
0810 07:22:34.990552 (+ 2117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:34.990557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:34.990634 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:34.990747 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:34.992521 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:34.993385 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:34.995134 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:34.995482 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:34.995583 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:34.998884 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:34.998889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:34.998892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:34.998921 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:35.000791 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:35.001446 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:35.001453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:35.001456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:35.001556 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:35.001975 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:35.001978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:35.001980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:35.002239 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:35.002350 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:35.003013 (+ 663us) server_negotiation.cc:299] Negotiation successful
0810 07:22:35.003162 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:22:35.004323 21366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:34.973123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38615 (local address 127.1.125.194:48683)
0810 07:22:34.988207 (+ 15084us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:34.988290 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:34.990254 (+ 1964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:34.991119 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:34.991130 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:34.991533 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:34.992293 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:34.992316 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:34.993561 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:34.993577 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:34.994969 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:34.994981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:34.995818 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:34.995823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:34.996021 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:34.996830 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:34.996852 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:34.998678 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:35.000918 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:35.000924 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:35.000940 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:35.001290 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:35.001664 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:35.001667 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:35.001669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:35.001857 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:35.002379 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:35.002389 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:35.002721 (+ 332us) client_negotiation.cc:769] Sending connection context
0810 07:22:35.002945 (+ 224us) client_negotiation.cc:240] Negotiation successful
0810 07:22:35.003282 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14779,"thread_start_us":154,"threads_started":1}
I20260810 07:22:35.005827 21363 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38615
I20260810 07:22:35.006106 21363 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:35.006681 21363 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
W20260810 07:22:35.008433 21363 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:38615 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:22:35.008620 21363 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:35.008790 21363 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
I20260810 07:22:35.009773 21363 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:35.009965 21363 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
W20260810 07:22:35.010831 21363 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:22:35.120282 21365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:35.120687 21365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:35.120775 21365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:35.125923 21365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:35.126101 21365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:35.126250 21365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:22:35.133502 21365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.125.212:38223/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:37635
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=40213
--tserver_master_addrs=127.1.125.254:38615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:35.134893 21365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:35.136096 21365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:35.146457 21378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:35.146457 21379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:35.146958 21365 server_base.cc:1034] running on GCE node
W20260810 07:22:35.147240 21381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:35.147688 21365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:35.148360 21365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:35.149588 21365 hybrid_clock.cc:648] HybridClock initialized: now 1786346555149546 us; error 48 us; skew 500 ppm
Aug 10 07:22:35 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346555, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:35.152684 21365 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:22:35.153713 21365 webserver.cc:460] Webserver started at http://127.1.125.195:40213/ using document root <none> and password file <none>
I20260810 07:22:35.154299 21365 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:35.154409 21365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:22:35 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346555, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:22:35.173914 21365 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 07:22:35.177073 21394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:35.178529 21365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:22:35.178681 21365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "114dbfae2aec427cb5173f9297fa9644"
format_stamp: "Formatted at 2026-08-10 07:22:33 on dist-test-slave-44d2"
server_key: "25cb843bc62fbc4ebfc04db796f1b039"
server_key_iv: "6ed45f099a14bbb60bece6c2b8e01932"
server_key_version: "kuduclusterkey@0"
I20260810 07:22:35.179280 21365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:35.194044 21365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:35.197681 21365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:35.197958 21365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:35.198767 21365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:35.199865 21365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:35.199944 21365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:35.200073 21365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:35.200141 21365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260810 07:22:35.211831 21365 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:37635
I20260810 07:22:35.213238 21365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:22:35.214041 21583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:37635 every 8 connection(s)
I20260810 07:22:35.222591 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 21365
Aug 10 07:22:35 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346555, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:35.232357 21162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:35.215770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:42035 (local address 127.1.125.254:38615)
0810 07:22:35.215925 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:35.215931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:35.218915 (+ 2984us) server_negotiation.cc:407] Connection header received
0810 07:22:35.219916 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:35.219921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:35.219984 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:35.220065 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:35.221330 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:35.222213 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:35.225022 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:35.225320 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:35.225446 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:35.227928 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:35.227932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:35.227935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:35.227969 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:35.229662 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:35.230286 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:35.230290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:35.230292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:35.230390 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:35.230819 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:35.230824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:35.230825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:35.231069 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:35.231186 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:35.232099 (+ 913us) server_negotiation.cc:299] Negotiation successful
0810 07:22:35.232227 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 07:22:35.233636 21587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:35.216296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38615 (local address 127.1.125.195:42035)
0810 07:22:35.218749 (+ 2453us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:35.218793 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:35.219650 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:35.220255 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:35.220262 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:35.220616 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:35.221126 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:35.221140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:35.223837 (+ 2697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:35.223843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:35.224882 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:35.224894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:35.225489 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:35.225493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:35.225683 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:35.226310 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:35.226327 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:35.227735 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:35.229804 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:35.229811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:35.229828 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:35.230155 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:35.230503 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:35.230506 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:35.230508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:35.230700 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:35.231247 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:35.231254 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:35.231600 (+ 346us) client_negotiation.cc:769] Sending connection context
0810 07:22:35.231812 (+ 212us) client_negotiation.cc:240] Negotiation successful
0810 07:22:35.232845 (+ 1033us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2250,"thread_start_us":216,"threads_started":1}
I20260810 07:22:35.235250 21584 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38615
I20260810 07:22:35.235517 21584 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:35.236052 21584 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
W20260810 07:22:35.238169 21584 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:38615 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:22:35.238327 21584 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:35.238564 21584 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
I20260810 07:22:35.239481 21584 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:35.239995 21584 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
W20260810 07:22:35.241353 21584 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:22:35.242983 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:35.223800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56000 (local address 127.1.125.254:38615)
0810 07:22:35.225158 (+ 1358us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:35.225162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:35.234025 (+ 8863us) server_negotiation.cc:407] Connection header received
0810 07:22:35.234076 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:35.234091 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:35.234147 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:35.234262 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:35.235240 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:35.236147 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:35.237202 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:35.237498 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:35.237604 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:35.239229 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:35.239233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:35.239236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:35.239266 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:35.241184 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:35.241691 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:35.241695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:35.241697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:35.241792 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:35.242099 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:35.242106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:35.242110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:35.242299 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:35.242542 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:35.242671 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 07:22:35.242808 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1269,"thread_start_us":103,"threads_started":1}
Aug 10 07:22:35 dist-test-slave-44d2 krb5kdc[19331](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346555, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:35.732445 21158 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:35.732913 21158 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
I20260810 07:22:35.734104 20847 ts_manager.cc:194] Registered new tserver with Master: f74abe29c92a4888afd4c04adcc5a586 (127.1.125.193:44077)
I20260810 07:22:35.735518 20847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:57993
I20260810 07:22:36.011152 21363 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:36.011503 21363 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
I20260810 07:22:36.012370 20846 ts_manager.cc:194] Registered new tserver with Master: aaf73521d83c4e60835c72b7ff03e392 (127.1.125.194:34865)
I20260810 07:22:36.013217 20846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:48683
Aug 10 07:22:36 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346555, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
I20260810 07:22:36.241585 21584 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:36.241904 21584 heartbeater.cc:511] Master 127.1.125.254:38615 requested a full tablet report, sending...
I20260810 07:22:36.242668 20846 ts_manager.cc:194] Registered new tserver with Master: 114dbfae2aec427cb5173f9297fa9644 (127.1.125.195:37635)
I20260810 07:22:36.243314 20846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:42035
I20260810 07:22:36.250010 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:22:36.260633 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:36.251791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56004 (local address 127.1.125.254:38615)
0810 07:22:36.252052 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:36.252057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:36.252279 (+ 222us) server_negotiation.cc:407] Connection header received
0810 07:22:36.252315 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:36.252317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:36.252356 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:36.252452 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:36.253212 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.253764 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.254676 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.254931 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.255048 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:36.256535 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:36.256539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:36.256542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:36.256579 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:36.258602 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:36.259144 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:36.259147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:36.259149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:36.259249 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:36.259602 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:36.259606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:36.259607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:36.259850 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:36.259997 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:36.260247 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 07:22:36.260375 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":80,"threads_started":1}
I20260810 07:22:36.264271 20846 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:22:36.268018 20846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:22:36.286921 21630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:36.279727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47078 (local address 127.1.125.193:44077)
0810 07:22:36.280139 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:36.280147 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:36.280938 (+ 791us) server_negotiation.cc:407] Connection header received
0810 07:22:36.281177 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:36.281183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:36.281394 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:36.281519 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:36.282176 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.283362 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.285120 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.285959 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.286047 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:36.286210 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:36.286660 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 07:22:36.286762 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":97,"threads_started":1}
I20260810 07:22:36.287703 21628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:36.279097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:44077 (local address 127.0.0.1:47078)
0810 07:22:36.280842 (+ 1745us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:36.280854 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:36.280993 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:36.281740 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:36.281745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:36.281784 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:36.282026 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:36.282034 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.283620 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.283623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:36.284978 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:36.284989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.286307 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.286311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:36.286428 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:36.286472 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 07:22:36.287382 (+ 910us) client_negotiation.cc:240] Negotiation successful
0810 07:22:36.287522 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1674,"mutex_wait_us":74,"spinlock_wait_cycles":8960,"thread_start_us":166,"threads_started":1}
I20260810 07:22:36.287962 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:36.279728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47830 (local address 127.1.125.195:37635)
0810 07:22:36.280364 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:36.280371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:36.280859 (+ 488us) server_negotiation.cc:407] Connection header received
0810 07:22:36.281177 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:36.281183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:36.281393 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:36.281519 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:36.282176 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.284065 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.285544 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.287070 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.287167 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:36.287307 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:36.287651 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 07:22:36.287760 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":100,"threads_started":1}
I20260810 07:22:36.288594 21627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:36.278850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:37635 (local address 127.0.0.1:47830)
0810 07:22:36.280710 (+ 1860us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:36.280766 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:36.280978 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:36.281740 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:36.281745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:36.281782 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:36.282026 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:36.282034 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.284312 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.284315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:36.285411 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:36.285420 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.287404 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.287407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:36.287491 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:36.287511 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 07:22:36.288302 (+ 791us) client_negotiation.cc:240] Negotiation successful
0810 07:22:36.288380 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1679,"mutex_wait_us":23,"spinlock_wait_cycles":6528,"thread_start_us":111,"threads_started":1}
I20260810 07:22:36.291518 21048 tablet_service.cc:1467] Processing CreateTablet for tablet 10707f86945f4574a567a313d9eb7188 (DEFAULT_TABLE table=test-table [id=c788f07138f142a89b89d83a939b6641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:36.292798 21048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10707f86945f4574a567a313d9eb7188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:36.293283 21628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:36.279192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:34865 (local address 127.0.0.1:47830)
0810 07:22:36.289151 (+ 9959us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:36.289167 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:36.289329 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:36.290060 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:36.290063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:36.290080 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:36.290150 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:36.290156 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.291211 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.291213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:36.291893 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:36.291899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.292829 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.292833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:36.292986 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:36.293020 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 07:22:36.293099 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:22:36.293168 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9901,"thread_start_us":112,"threads_started":1}
I20260810 07:22:36.293419 21629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:36.279364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47830 (local address 127.1.125.194:34865)
0810 07:22:36.280696 (+ 1332us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:36.280702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:36.289258 (+ 8556us) server_negotiation.cc:407] Connection header received
0810 07:22:36.289477 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:36.289482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:36.289709 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:36.289844 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:36.290289 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.291093 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.292030 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.292701 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.292819 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:36.292960 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:36.293159 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 07:22:36.293253 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1233,"thread_start_us":117,"threads_started":1}
I20260810 07:22:36.294934 21439 tablet_service.cc:1467] Processing CreateTablet for tablet 10707f86945f4574a567a313d9eb7188 (DEFAULT_TABLE table=test-table [id=c788f07138f142a89b89d83a939b6641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:36.295955 21439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10707f86945f4574a567a313d9eb7188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:36.297063 21263 tablet_service.cc:1467] Processing CreateTablet for tablet 10707f86945f4574a567a313d9eb7188 (DEFAULT_TABLE table=test-table [id=c788f07138f142a89b89d83a939b6641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:36.298023 21263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10707f86945f4574a567a313d9eb7188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:36.305094 21632 tablet_bootstrap.cc:492] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586: Bootstrap starting.
I20260810 07:22:36.307552 21632 tablet_bootstrap.cc:654] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:36.307967 21633 tablet_bootstrap.cc:492] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644: Bootstrap starting.
I20260810 07:22:36.310209 21633 tablet_bootstrap.cc:654] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:36.311054 21633 log.cc:824] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:36.312249 21632 log.cc:824] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:36.313249 21634 tablet_bootstrap.cc:492] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392: Bootstrap starting.
I20260810 07:22:36.314857 21633 tablet_bootstrap.cc:492] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644: No bootstrap required, opened a new log
I20260810 07:22:36.315050 21633 ts_tablet_manager.cc:1397] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260810 07:22:36.316535 21634 tablet_bootstrap.cc:654] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:36.317375 21634 log.cc:824] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:36.319380 21632 tablet_bootstrap.cc:492] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586: No bootstrap required, opened a new log
I20260810 07:22:36.319571 21632 ts_tablet_manager.cc:1397] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20260810 07:22:36.319722 21634 tablet_bootstrap.cc:492] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392: No bootstrap required, opened a new log
I20260810 07:22:36.319900 21634 ts_tablet_manager.cc:1397] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260810 07:22:36.324128 21632 raft_consensus.cc:348] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114dbfae2aec427cb5173f9297fa9644" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 37635 } } peers { permanent_uuid: "aaf73521d83c4e60835c72b7ff03e392" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34865 } } peers { permanent_uuid: "f74abe29c92a4888afd4c04adcc5a586" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44077 } }
I20260810 07:22:36.324208 21634 raft_consensus.cc:348] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114dbfae2aec427cb5173f9297fa9644" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 37635 } } peers { permanent_uuid: "aaf73521d83c4e60835c72b7ff03e392" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34865 } } peers { permanent_uuid: "f74abe29c92a4888afd4c04adcc5a586" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44077 } }
I20260810 07:22:36.324468 21634 raft_consensus.cc:374] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:36.324581 21634 raft_consensus.cc:729] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaf73521d83c4e60835c72b7ff03e392, State: Initialized, Role: FOLLOWER
I20260810 07:22:36.325057 21634 consensus_queue.cc:260] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114dbfae2aec427cb5173f9297fa9644" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 37635 } } peers { permanent_uuid: "aaf73521d83c4e60835c72b7ff03e392" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34865 } } peers { permanent_uuid: "f74abe29c92a4888afd4c04adcc5a586" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44077 } }
I20260810 07:22:36.326133 21633 raft_consensus.cc:348] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114dbfae2aec427cb5173f9297fa9644" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 37635 } } peers { permanent_uuid: "aaf73521d83c4e60835c72b7ff03e392" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34865 } } peers { permanent_uuid: "f74abe29c92a4888afd4c04adcc5a586" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44077 } }
I20260810 07:22:36.326450 21632 raft_consensus.cc:374] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:36.326550 21632 raft_consensus.cc:729] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74abe29c92a4888afd4c04adcc5a586, State: Initialized, Role: FOLLOWER
I20260810 07:22:36.327112 21634 ts_tablet_manager.cc:1428] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260810 07:22:36.327256 21633 raft_consensus.cc:374] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:36.327116 21632 consensus_queue.cc:260] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114dbfae2aec427cb5173f9297fa9644" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 37635 } } peers { permanent_uuid: "aaf73521d83c4e60835c72b7ff03e392" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34865 } } peers { permanent_uuid: "f74abe29c92a4888afd4c04adcc5a586" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44077 } }
I20260810 07:22:36.327509 21363 heartbeater.cc:503] Master 127.1.125.254:38615 was elected leader, sending a full tablet report...
I20260810 07:22:36.327517 21633 raft_consensus.cc:729] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 114dbfae2aec427cb5173f9297fa9644, State: Initialized, Role: FOLLOWER
I20260810 07:22:36.328544 21632 ts_tablet_manager.cc:1428] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260810 07:22:36.329056 21633 consensus_queue.cc:260] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114dbfae2aec427cb5173f9297fa9644" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 37635 } } peers { permanent_uuid: "aaf73521d83c4e60835c72b7ff03e392" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34865 } } peers { permanent_uuid: "f74abe29c92a4888afd4c04adcc5a586" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44077 } }
I20260810 07:22:36.330045 21633 ts_tablet_manager.cc:1428] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260810 07:22:36.330416 21158 heartbeater.cc:503] Master 127.1.125.254:38615 was elected leader, sending a full tablet report...
I20260810 07:22:36.332808 21584 heartbeater.cc:503] Master 127.1.125.254:38615 was elected leader, sending a full tablet report...
I20260810 07:22:36.343237 21641 raft_consensus.cc:482] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:36.343421 21641 raft_consensus.cc:504] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114dbfae2aec427cb5173f9297fa9644" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 37635 } } peers { permanent_uuid: "aaf73521d83c4e60835c72b7ff03e392" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34865 } } peers { permanent_uuid: "f74abe29c92a4888afd4c04adcc5a586" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44077 } }
I20260810 07:22:36.346010 21641 leader_election.cc:288] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aaf73521d83c4e60835c72b7ff03e392 (127.1.125.194:34865), f74abe29c92a4888afd4c04adcc5a586 (127.1.125.193:44077)
I20260810 07:22:36.350030 21629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:36.345343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:52283 (local address 127.1.125.194:34865)
0810 07:22:36.345480 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:36.345485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:36.345510 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:22:36.345711 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:36.345715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:36.345769 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:36.345827 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:36.346310 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.347091 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.348748 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.349320 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.349404 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:36.349453 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:36.349807 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 07:22:36.349885 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 07:22:36.350509 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:36.344968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:34865 (local address 127.1.125.195:52283)
0810 07:22:36.345424 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:36.345440 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:36.345560 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:36.345986 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:36.345989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:36.346011 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:36.346169 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:36.346177 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.347243 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.347246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:36.348612 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:36.348623 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.349556 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.349560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:36.349658 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:36.349682 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:22:36.350210 (+ 528us) client_negotiation.cc:240] Negotiation successful
0810 07:22:36.350313 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260810 07:22:36.350651 21630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:36.345845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:43259 (local address 127.1.125.193:44077)
0810 07:22:36.345967 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:36.345971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:36.346388 (+ 417us) server_negotiation.cc:407] Connection header received
0810 07:22:36.346558 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:36.346561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:36.346606 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:36.346650 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:36.347006 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.347725 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.349146 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.349728 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.349818 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:36.349859 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:36.350437 (+ 578us) server_negotiation.cc:299] Negotiation successful
0810 07:22:36.350515 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:22:36.351290 21294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10707f86945f4574a567a313d9eb7188" candidate_uuid: "114dbfae2aec427cb5173f9297fa9644" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaf73521d83c4e60835c72b7ff03e392" is_pre_election: true
I20260810 07:22:36.351619 21294 raft_consensus.cc:2454] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 114dbfae2aec427cb5173f9297fa9644 in term 0.
I20260810 07:22:36.352125 21645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:36.345491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:44077 (local address 127.1.125.195:43259)
0810 07:22:36.346308 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:36.346321 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:36.346448 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:36.346768 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:36.346771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:36.346785 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:36.346889 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:36.346894 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.347864 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.347867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:36.349018 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:36.349028 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.350000 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.350003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:36.350291 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:36.350313 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:22:36.350947 (+ 634us) client_negotiation.cc:240] Negotiation successful
0810 07:22:36.351980 (+ 1033us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":97,"threads_started":1}
I20260810 07:22:36.352128 21404 leader_election.cc:302] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114dbfae2aec427cb5173f9297fa9644, aaf73521d83c4e60835c72b7ff03e392; no voters:
I20260810 07:22:36.352792 21641 raft_consensus.cc:2781] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:22:36.352888 21641 raft_consensus.cc:482] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:22:36.352942 21641 raft_consensus.cc:3037] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:36.352985 21074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10707f86945f4574a567a313d9eb7188" candidate_uuid: "114dbfae2aec427cb5173f9297fa9644" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74abe29c92a4888afd4c04adcc5a586" is_pre_election: true
I20260810 07:22:36.353268 21074 raft_consensus.cc:2454] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 114dbfae2aec427cb5173f9297fa9644 in term 0.
I20260810 07:22:36.354890 21641 raft_consensus.cc:504] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114dbfae2aec427cb5173f9297fa9644" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 37635 } } peers { permanent_uuid: "aaf73521d83c4e60835c72b7ff03e392" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34865 } } peers { permanent_uuid: "f74abe29c92a4888afd4c04adcc5a586" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44077 } }
I20260810 07:22:36.355630 21294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10707f86945f4574a567a313d9eb7188" candidate_uuid: "114dbfae2aec427cb5173f9297fa9644" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaf73521d83c4e60835c72b7ff03e392"
I20260810 07:22:36.355777 21294 raft_consensus.cc:3037] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:36.357061 21294 raft_consensus.cc:2454] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 114dbfae2aec427cb5173f9297fa9644 in term 1.
I20260810 07:22:36.357365 21074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10707f86945f4574a567a313d9eb7188" candidate_uuid: "114dbfae2aec427cb5173f9297fa9644" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74abe29c92a4888afd4c04adcc5a586"
I20260810 07:22:36.357506 21074 raft_consensus.cc:3037] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:36.357597 21404 leader_election.cc:302] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114dbfae2aec427cb5173f9297fa9644, aaf73521d83c4e60835c72b7ff03e392; no voters:
I20260810 07:22:36.358728 21074 raft_consensus.cc:2454] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 114dbfae2aec427cb5173f9297fa9644 in term 1.
I20260810 07:22:36.358960 21641 leader_election.cc:288] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [CANDIDATE]: Term 1 election: Requested vote from peers aaf73521d83c4e60835c72b7ff03e392 (127.1.125.194:34865), f74abe29c92a4888afd4c04adcc5a586 (127.1.125.193:44077)
I20260810 07:22:36.359179 21641 raft_consensus.cc:2781] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:36.359314 21641 raft_consensus.cc:686] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [term 1 LEADER]: Becoming Leader. State: Replica: 114dbfae2aec427cb5173f9297fa9644, State: Running, Role: LEADER
I20260810 07:22:36.359647 21641 consensus_queue.cc:237] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114dbfae2aec427cb5173f9297fa9644" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 37635 } } peers { permanent_uuid: "aaf73521d83c4e60835c72b7ff03e392" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34865 } } peers { permanent_uuid: "f74abe29c92a4888afd4c04adcc5a586" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44077 } }
I20260810 07:22:36.362717 20846 catalog_manager.cc:5495] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 reported cstate change: term changed from 0 to 1, leader changed from <none> to 114dbfae2aec427cb5173f9297fa9644 (127.1.125.195). New cstate: current_term: 1 leader_uuid: "114dbfae2aec427cb5173f9297fa9644" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114dbfae2aec427cb5173f9297fa9644" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 37635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aaf73521d83c4e60835c72b7ff03e392" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f74abe29c92a4888afd4c04adcc5a586" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44077 } health_report { overall_health: UNKNOWN } } }
I20260810 07:22:36.383859 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:36.378953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.1.125.195:37635)
0810 07:22:36.379102 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:36.379106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:36.379120 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:22:36.379199 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:36.379202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:36.379253 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:36.379372 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:22:36.379659 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.380194 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.381446 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:36.381809 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:36.381916 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:36.382351 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:22:36.382959 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:22:36.383088 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:36.383599 (+ 511us) server_negotiation.cc:299] Negotiation successful
0810 07:22:36.383669 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:22:36.391556 21294 raft_consensus.cc:1261] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392 [term 1 FOLLOWER]: Refusing update from remote peer 114dbfae2aec427cb5173f9297fa9644: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:36.391556 21074 raft_consensus.cc:1261] T 10707f86945f4574a567a313d9eb7188 P f74abe29c92a4888afd4c04adcc5a586 [term 1 FOLLOWER]: Refusing update from remote peer 114dbfae2aec427cb5173f9297fa9644: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:36.392388 21641 consensus_queue.cc:1035] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f74abe29c92a4888afd4c04adcc5a586" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 44077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:36.392876 21641 consensus_queue.cc:1035] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaf73521d83c4e60835c72b7ff03e392" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 34865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:36.403091 21651 mvcc.cc:205] Tried to move back new op lower bound from 7316875494970142720 to 7316875494852083712. Current Snapshot: MvccSnapshot[applied={T|T < 7316875494970142720}]
I20260810 07:22:36.403091 21653 mvcc.cc:205] Tried to move back new op lower bound from 7316875494970142720 to 7316875494852083712. Current Snapshot: MvccSnapshot[applied={T|T < 7316875494970142720}]
I20260810 07:22:36.406706 21652 mvcc.cc:205] Tried to move back new op lower bound from 7316875494970142720 to 7316875494852083712. Current Snapshot: MvccSnapshot[applied={T|T < 7316875494970142720}]
I20260810 07:22:36.416671 20847 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:36.416853 20847 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:36.420055 20847 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e72daf618e804bab849652a057827fb4: Sending DeleteTablet for 3 replicas of tablet 10707f86945f4574a567a313d9eb7188
I20260810 07:22:36.421216 21263 tablet_service.cc:1514] Processing DeleteTablet for tablet 10707f86945f4574a567a313d9eb7188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:36 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:47830
I20260810 07:22:36.422279 21048 tablet_service.cc:1514] Processing DeleteTablet for tablet 10707f86945f4574a567a313d9eb7188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:36 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:47078
I20260810 07:22:36.423143 21439 tablet_service.cc:1514] Processing DeleteTablet for tablet 10707f86945f4574a567a313d9eb7188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:36 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:47830
I20260810 07:22:36.424433 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 20912
I20260810 07:22:36.432755 21669 tablet_replica.cc:330] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392: stopping tablet replica
I20260810 07:22:36.433184 21669 raft_consensus.cc:2229] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:36.433569 21669 raft_consensus.cc:2258] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:36.434181 21673 tablet_replica.cc:330] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644: stopping tablet replica
I20260810 07:22:36.434466 21673 raft_consensus.cc:2229] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:22:36.434880 21673 raft_consensus.cc:2258] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:36.436175 21669 ts_tablet_manager.cc:1905] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:36.439285 21669 ts_tablet_manager.cc:1918] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:36.440841 21673 ts_tablet_manager.cc:1905] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:36.443517 21673 ts_tablet_manager.cc:1918] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:36.443704 21669 log.cc:1196] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/10707f86945f4574a567a313d9eb7188
I20260810 07:22:36.444128 21669 ts_tablet_manager.cc:1939] T 10707f86945f4574a567a313d9eb7188 P aaf73521d83c4e60835c72b7ff03e392: Deleting consensus metadata
W20260810 07:22:36.445449 20818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:44077 (error 108)
Aug 10 07:22:36 dist-test-slave-44d2 krb5kdc[19331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346555, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for HTTP/127.1.125.212@KRBTEST.COM
W20260810 07:22:36.447479 21628 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:22:36.446738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:44077 (local address 127.0.0.1:47082)
0810 07:22:36.446927 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:36.446943 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:36.447053 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:36.447368 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:44077: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":110}
W20260810 07:22:36.447772 20818 catalog_manager.cc:4584] TS f74abe29c92a4888afd4c04adcc5a586 (127.1.125.193:44077): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 10707f86945f4574a567a313d9eb7188: Network error: Client connection negotiation failed: client connection to 127.1.125.193:44077: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:22:36.448459 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 21161
I20260810 07:22:36.455580 21673 log.cc:1196] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/10707f86945f4574a567a313d9eb7188
I20260810 07:22:36.458037 21673 ts_tablet_manager.cc:1939] T 10707f86945f4574a567a313d9eb7188 P 114dbfae2aec427cb5173f9297fa9644: Deleting consensus metadata
I20260810 07:22:36.459509 20821 catalog_manager.cc:4841] TS 114dbfae2aec427cb5173f9297fa9644 (127.1.125.195:37635): tablet 10707f86945f4574a567a313d9eb7188 (table test-table [id=c788f07138f142a89b89d83a939b6641]) successfully deleted
W20260810 07:22:36.474852 21628 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:22:36.473102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:34865 (local address 127.0.0.1:47832)
0810 07:22:36.473301 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:36.473317 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:36.473410 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:36.474734 (+ 1324us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.194:34865: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":115}
I20260810 07:22:36.475721 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 21365
I20260810 07:22:36.494590 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 20792
I20260810 07:22:36.517683 1527 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 07:22:36.873904 1527 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 07:22:36.874028 1527 mini_ranger.cc:66] Stopping Ranger...
I20260810 07:22:37.229315 1527 mini_ranger.cc:68] Stopped Ranger
2026-08-10 07:22:37.229 UTC [19376] LOG: received smart shutdown request
2026-08-10 07:22:37.234 UTC [19376] LOG: background worker "logical replication launcher" (PID 19388) exited with exit code 1
2026-08-10 07:22:37.234 UTC [19381] LOG: shutting down
2026-08-10 07:22:37.393 UTC [19376] LOG: database system is shut down
2026-08-10T07:22:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50999 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:22:37 dist-test-slave-44d2 krb5kdc[21690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:22:37 dist-test-slave-44d2 krb5kdc[21690](info): set up 2 sockets
Aug 10 07:22:37 dist-test-slave-44d2 krb5kdc[21690](info): commencing operation
krb5kdc: starting...
W20260810 07:22:38.874706 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:22:38 dist-test-slave-44d2 krb5kdc[21690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:22:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:22:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:22:39.007678 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:45701
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:45993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:45701,127.1.125.253:37293,127.1.125.252:43497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:39.123538 21708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:39.123832 21708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:39.123914 21708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:39.127887 21708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:39.127974 21708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:39.128010 21708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:39.128069 21708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:39.128108 21708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:39.133515 21708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:45701,127.1.125.253:37293,127.1.125.252:43497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:45701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:39.134667 21708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:39.135749 21708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:39.145151 21719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:39.145128 21717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:39.145169 21721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:39.145574 21708 server_base.cc:1034] running on GCE node
I20260810 07:22:39.146162 21708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:39.147284 21708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:39.148517 21708 hybrid_clock.cc:648] HybridClock initialized: now 1786346559148450 us; error 49 us; skew 500 ppm
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:39.151259 21708 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:22:39.152153 21708 webserver.cc:460] Webserver started at http://127.1.125.254:37329/ using document root <none> and password file <none>
I20260810 07:22:39.152765 21708 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:39.152834 21708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:39.153012 21708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:39.155071 21708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "a4cbf14818b1452b884b81a4345772c7"
format_stamp: "Formatted at 2026-08-10 07:22:39 on dist-test-slave-44d2"
server_key: "88b77314616961f280bca8231f88c498"
server_key_iv: "e9288b94af9b062ccb697a84830c5702"
server_key_version: "encryptionkey@0"
I20260810 07:22:39.155594 21708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "a4cbf14818b1452b884b81a4345772c7"
format_stamp: "Formatted at 2026-08-10 07:22:39 on dist-test-slave-44d2"
server_key: "88b77314616961f280bca8231f88c498"
server_key_iv: "e9288b94af9b062ccb697a84830c5702"
server_key_version: "encryptionkey@0"
I20260810 07:22:39.159255 21708 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 07:22:39.161680 21733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:39.162905 21708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:22:39.163038 21708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "a4cbf14818b1452b884b81a4345772c7"
format_stamp: "Formatted at 2026-08-10 07:22:39 on dist-test-slave-44d2"
server_key: "88b77314616961f280bca8231f88c498"
server_key_iv: "e9288b94af9b062ccb697a84830c5702"
server_key_version: "encryptionkey@0"
I20260810 07:22:39.163141 21708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:39.185858 21708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:39.188884 21708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:39.189126 21708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:39.196815 21708 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:45701
I20260810 07:22:39.196846 21811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:45701 every 8 connection(s)
I20260810 07:22:39.197800 21708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:39.200933 21812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:39.204856 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 21708
I20260810 07:22:39.204973 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:22:39.205260 1527 external_mini_cluster.cc:1428] Setting key a29d593e4b434bd8aa96820935a2eeb2
I20260810 07:22:39.204974 21812 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.125.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.253@KRBTEST.COM" created.
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:39.226331 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.208857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46120 (local address 127.1.125.254:45701)
0810 07:22:39.209656 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.209668 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.209714 (+ 46us) server_negotiation.cc:407] Connection header received
0810 07:22:39.210246 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.210255 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.210498 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.210696 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.212410 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.214021 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.215611 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.216284 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.216449 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.220706 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.220717 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.220736 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.220773 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.222923 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.223459 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.223465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.223466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.223593 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.223917 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.223922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.223924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.224316 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.224468 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.224945 (+ 477us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.225195 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":696,"thread_start_us":87,"threads_started":1}
I20260810 07:22:39.226327 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.207844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45701 (local address 127.0.0.1:46120)
0810 07:22:39.208447 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.208550 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:39.209826 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:39.211258 (+ 1432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:39.211268 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:39.211822 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:39.212228 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.212254 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.214166 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.214172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.215448 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.215463 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.216709 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.216715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.216847 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.217550 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:39.217569 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:39.220428 (+ 2859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:39.223059 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.223067 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.223077 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.223337 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.223691 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.223696 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.223698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.223818 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.224437 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:39.224444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:39.224648 (+ 204us) client_negotiation.cc:769] Sending connection context
0810 07:22:39.224876 (+ 228us) client_negotiation.cc:240] Negotiation successful
0810 07:22:39.225261 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":100,"thread_start_us":118,"threads_started":1}
I20260810 07:22:39.229983 21812 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:22:39.230808 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.230542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:37293 (local address 127.0.0.1:55704)
0810 07:22:39.230666 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.230738 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:37293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 07:22:39.231056 21738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.125.253:37293: connect: Connection refused (error 111)
I20260810 07:22:39.231926 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.231709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:37293 (local address 127.0.0.1:55720)
0810 07:22:39.231820 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.231869 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:37293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 07:22:39.232105 21812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.253:37293: Network error: Client connection negotiation failed: client connection to 127.1.125.253:37293: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.253.keytab.
Entry for principal kudu/127.1.125.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.253.keytab.
I20260810 07:22:39.237349 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.253:37293
--webserver_interface=127.1.125.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.253.keytab
--principal=kudu/127.1.125.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:45993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:45701,127.1.125.253:37293,127.1.125.252:43497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:39.281363 21812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } attempt: 1
I20260810 07:22:39.282430 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.281981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:37293 (local address 127.0.0.1:55728)
0810 07:22:39.282203 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.282302 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:37293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 07:22:39.283380 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.283123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:37293 (local address 127.0.0.1:55732)
0810 07:22:39.283243 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.283301 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:37293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 07:22:39.283567 21812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.253:37293: Network error: Client connection negotiation failed: client connection to 127.1.125.253:37293: connect: Connection refused (error 111)
I20260810 07:22:39.351769 21812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } attempt: 2
I20260810 07:22:39.352696 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.352331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:37293 (local address 127.0.0.1:55738)
0810 07:22:39.352535 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.352618 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:37293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 07:22:39.353549 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.353296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:37293 (local address 127.0.0.1:55752)
0810 07:22:39.353410 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.353460 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:37293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 07:22:39.353724 21812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.253:37293: Network error: Client connection negotiation failed: client connection to 127.1.125.253:37293: connect: Connection refused (error 111)
W20260810 07:22:39.355571 21826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:39.355808 21826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:39.355923 21826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:39.359925 21826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:39.359979 21826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:39.360034 21826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:39.360071 21826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:39.360129 21826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:39.365482 21826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:45701,127.1.125.253:37293,127.1.125.252:43497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.253.keytab
--principal=kudu/127.1.125.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.253:37293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.125.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:39.366580 21826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:39.367640 21826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:39.376523 21834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:39.376572 21835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:39.376577 21838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:39.376729 21826 server_base.cc:1034] running on GCE node
I20260810 07:22:39.377367 21826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:39.378474 21826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:39.379676 21826 hybrid_clock.cc:648] HybridClock initialized: now 1786346559379662 us; error 28 us; skew 500 ppm
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:39.382905 21826 init.cc:377] Logged in from keytab as kudu/127.1.125.253@KRBTEST.COM (short username kudu)
I20260810 07:22:39.383843 21826 webserver.cc:460] Webserver started at http://127.1.125.253:44307/ using document root <none> and password file <none>
I20260810 07:22:39.384429 21826 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:39.384562 21826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:39.384758 21826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:39.387140 21826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/data/instance:
uuid: "a1c595a55ce64ce5a7dce272e16e8373"
format_stamp: "Formatted at 2026-08-10 07:22:39 on dist-test-slave-44d2"
server_key: "9076b03e29cb5724afb1c5c17ac2b56c"
server_key_iv: "29aba93184b510f96b211069dc53861c"
server_key_version: "encryptionkey@0"
I20260810 07:22:39.387725 21826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/wal/instance:
uuid: "a1c595a55ce64ce5a7dce272e16e8373"
format_stamp: "Formatted at 2026-08-10 07:22:39 on dist-test-slave-44d2"
server_key: "9076b03e29cb5724afb1c5c17ac2b56c"
server_key_iv: "29aba93184b510f96b211069dc53861c"
server_key_version: "encryptionkey@0"
I20260810 07:22:39.391531 21826 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260810 07:22:39.394047 21849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:39.395272 21826 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:22:39.395427 21826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/wal
uuid: "a1c595a55ce64ce5a7dce272e16e8373"
format_stamp: "Formatted at 2026-08-10 07:22:39 on dist-test-slave-44d2"
server_key: "9076b03e29cb5724afb1c5c17ac2b56c"
server_key_iv: "29aba93184b510f96b211069dc53861c"
server_key_version: "encryptionkey@0"
I20260810 07:22:39.395522 21826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:39.409427 21826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:39.412143 21826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:39.412348 21826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:39.419580 21826 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.253:37293
I20260810 07:22:39.419638 21921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.253:37293 every 8 connection(s)
I20260810 07:22:39.420657 21826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/data/info.pb
I20260810 07:22:39.423411 21922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:39.424772 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 21826
I20260810 07:22:39.424919 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-1/wal/instance
I20260810 07:22:39.425211 1527 external_mini_cluster.cc:1428] Setting key ba5c9a1403e17d0e859befeb50e89f46
I20260810 07:22:39.427253 21922 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.125.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.252@KRBTEST.COM" created.
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.253@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:39.442919 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.429141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46126 (local address 127.1.125.254:45701)
0810 07:22:39.429306 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.429311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.430153 (+ 842us) server_negotiation.cc:407] Connection header received
0810 07:22:39.431072 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.431078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.431154 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.431264 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.432386 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.433316 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.434433 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.434897 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.434993 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.438549 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.438552 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.438554 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.438581 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.440381 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.440976 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.440982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.440986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.441084 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.441417 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.441424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.441428 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.441719 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.441852 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.442505 (+ 653us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.442649 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 07:22:39.443506 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.429514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45701 (local address 127.0.0.1:46126)
0810 07:22:39.429972 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.430025 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:39.430853 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:39.431423 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:39.431430 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:39.431764 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:39.432221 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.432234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.433447 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.433450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.434300 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.434310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.435042 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.435045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.435188 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.435772 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:39.435789 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:39.438357 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:39.440569 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.440577 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.440596 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.440864 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.441183 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.441186 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.441187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.441319 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.441860 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:39.441867 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:39.442215 (+ 348us) client_negotiation.cc:769] Sending connection context
0810 07:22:39.442386 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 07:22:39.442642 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":132,"threads_started":1}
I20260810 07:22:39.444890 21812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } attempt: 3
I20260810 07:22:39.445171 21922 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.253@KRBTEST.COM for kudu/127.1.125.253@KRBTEST.COM
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for kudu/127.1.125.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.252.keytab.
Entry for principal kudu/127.1.125.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.252.keytab.
I20260810 07:22:39.455391 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.252:43497
--webserver_interface=127.1.125.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.252.keytab
--principal=kudu/127.1.125.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:45993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:45701,127.1.125.253:37293,127.1.125.252:43497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:39.456548 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.445798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:37293 (local address 127.0.0.1:55766)
0810 07:22:39.445959 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.445972 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:39.446044 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:39.446575 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:39.446578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:39.446763 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:39.446844 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.446849 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.447930 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.447936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.448715 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.448726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.449516 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.449520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.449708 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.450315 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:39.450327 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:39.451717 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:39.454829 (+ 3112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.454832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.454837 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.455057 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.455374 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.455378 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.455380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.455493 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.456002 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:39.456005 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:39.456096 (+ 91us) client_negotiation.cc:769] Sending connection context
0810 07:22:39.456205 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 07:22:39.456338 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":47,"spinlock_wait_cycles":2304}
I20260810 07:22:39.456727 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.445332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:37293 (local address 127.0.0.1:55760)
0810 07:22:39.445440 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.445454 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:39.445529 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:39.446638 (+ 1109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:39.446644 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:39.446825 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:39.447324 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.447331 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.448272 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.448275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.448995 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.449004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.449589 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.449594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.449707 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.450387 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:39.450401 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:39.452413 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:39.454925 (+ 2512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.454928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.454930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.455210 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.455518 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.455522 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.455524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.455626 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.456166 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:39.456172 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:39.456279 (+ 107us) client_negotiation.cc:769] Sending connection context
0810 07:22:39.456387 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 07:22:39.456542 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 07:22:39.456831 21930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.445810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55766 (local address 127.1.125.253:37293)
0810 07:22:39.446217 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.446223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.446232 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:22:39.446255 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.446257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.446455 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.446600 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.446953 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.447822 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.448838 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.449396 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.449619 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.451899 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.451910 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.451924 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.451950 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.454710 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.455170 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.455176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.455178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.455278 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.455600 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.455603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.455605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.455893 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.456040 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.456307 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.456474 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":127,"server-negotiator.queue_time_us":361,"spinlock_wait_cycles":16896,"thread_start_us":104,"threads_started":1}
I20260810 07:22:39.457424 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.445576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55760 (local address 127.1.125.253:37293)
0810 07:22:39.446151 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.446166 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.446191 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:22:39.446233 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.446240 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.446451 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.447424 (+ 973us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.447538 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.448136 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.449101 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.449434 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.449583 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.452639 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.452643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.452644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.452666 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.454814 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.455316 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.455322 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.455325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.455412 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.455739 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.455744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.455749 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.456000 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.456147 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.457059 (+ 912us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.457197 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":457,"spinlock_wait_cycles":4224,"thread_start_us":139,"threads_started":1}
I20260810 07:22:39.458810 21812 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:22:39.459298 21922 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:22:39.459965 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.459735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36002)
0810 07:22:39.459849 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.459915 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 07:22:39.459968 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.459321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:35998)
0810 07:22:39.459494 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.459876 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 07:22:39.460539 21850 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
I20260810 07:22:39.461138 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.460569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36014)
0810 07:22:39.460943 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.461025 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":128,"threads_started":1}
W20260810 07:22:39.461508 21812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:43497: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
I20260810 07:22:39.461575 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.461326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36028)
0810 07:22:39.461452 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.461507 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 07:22:39.461766 21922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:43497: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
I20260810 07:22:39.492666 21812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } attempt: 1
I20260810 07:22:39.493651 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.493252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36044)
0810 07:22:39.493456 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.493539 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 07:22:39.494462 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.494210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36060)
0810 07:22:39.494333 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.494391 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 07:22:39.494620 21812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:43497: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
I20260810 07:22:39.511027 21922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } attempt: 1
I20260810 07:22:39.511934 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.511631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36074)
0810 07:22:39.511787 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.511853 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 07:22:39.512758 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.512524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36090)
0810 07:22:39.512639 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.512695 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 07:22:39.512933 21922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:43497: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
I20260810 07:22:39.569793 21812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } attempt: 2
I20260810 07:22:39.570711 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.570355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36094)
0810 07:22:39.570533 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.570617 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 07:22:39.571458 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.571221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36106)
0810 07:22:39.571337 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.571394 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 07:22:39.571602 21812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:43497: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
W20260810 07:22:39.575526 21932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:39.575759 21932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:39.575838 21932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:39.579842 21932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:39.579893 21932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:39.579913 21932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:39.579958 21932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:39.579988 21932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:39.581077 21922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } attempt: 2
I20260810 07:22:39.581845 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.581517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36122)
0810 07:22:39.581688 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.581760 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 07:22:39.582628 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.582370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36128)
0810 07:22:39.582505 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.582557 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 07:22:39.582777 21922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:43497: Network error: Client connection negotiation failed: client connection to 127.1.125.252:43497: connect: Connection refused (error 111)
I20260810 07:22:39.585783 21932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:45993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:45701,127.1.125.253:37293,127.1.125.252:43497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.252.keytab
--principal=kudu/127.1.125.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.252:43497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.125.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:39.586783 21932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:39.587770 21932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:39.596565 21946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:39.596598 21944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:39.596777 21932 server_base.cc:1034] running on GCE node
W20260810 07:22:39.596612 21943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:39.597385 21932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:39.598646 21932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:39.599838 21932 hybrid_clock.cc:648] HybridClock initialized: now 1786346559599809 us; error 46 us; skew 500 ppm
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:39.602710 21932 init.cc:377] Logged in from keytab as kudu/127.1.125.252@KRBTEST.COM (short username kudu)
I20260810 07:22:39.603698 21932 webserver.cc:460] Webserver started at http://127.1.125.252:37057/ using document root <none> and password file <none>
I20260810 07:22:39.604333 21932 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:39.604434 21932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:39.604709 21932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:39.606868 21932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/data/instance:
uuid: "531ea8882bf04c0aa914b3730ffa7c23"
format_stamp: "Formatted at 2026-08-10 07:22:39 on dist-test-slave-44d2"
server_key: "b660a0bf599e1b5c5e01cd4c07fa44cb"
server_key_iv: "28ebdeba8703b6de55594a0e4e5f9c28"
server_key_version: "encryptionkey@0"
I20260810 07:22:39.607506 21932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/wal/instance:
uuid: "531ea8882bf04c0aa914b3730ffa7c23"
format_stamp: "Formatted at 2026-08-10 07:22:39 on dist-test-slave-44d2"
server_key: "b660a0bf599e1b5c5e01cd4c07fa44cb"
server_key_iv: "28ebdeba8703b6de55594a0e4e5f9c28"
server_key_version: "encryptionkey@0"
I20260810 07:22:39.611503 21932 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 07:22:39.614101 21956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:39.615355 21932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:22:39.615509 21932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/wal
uuid: "531ea8882bf04c0aa914b3730ffa7c23"
format_stamp: "Formatted at 2026-08-10 07:22:39 on dist-test-slave-44d2"
server_key: "b660a0bf599e1b5c5e01cd4c07fa44cb"
server_key_iv: "28ebdeba8703b6de55594a0e4e5f9c28"
server_key_version: "encryptionkey@0"
I20260810 07:22:39.615628 21932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:39.628314 21932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:39.631289 21932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:39.631505 21932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:39.639137 21932 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.252:43497
I20260810 07:22:39.639155 22036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.252:43497 every 8 connection(s)
I20260810 07:22:39.640126 21932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/data/info.pb
I20260810 07:22:39.643002 22037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:39.643497 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 21932
I20260810 07:22:39.643644 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346386457763-1527-0/minicluster-data/master-2/wal/instance
I20260810 07:22:39.643971 1527 external_mini_cluster.cc:1428] Setting key 9c4a8a9573b43176742be7662dd06ee1
I20260810 07:22:39.647929 22037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.252@KRBTEST.COM
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.252@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:39.659384 22040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.645420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36142 (local address 127.1.125.252:43497)
0810 07:22:39.645870 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.645886 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.645923 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:22:39.646605 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.646622 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.646972 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.647313 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.648181 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.649349 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.650435 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.650890 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.651015 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.654022 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.654027 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.654036 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.654064 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.655868 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.656431 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.656437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.656443 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.656616 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.656968 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.656974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.656980 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.657612 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.657851 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.658055 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.658365 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":131,"threads_started":1}
I20260810 07:22:39.660697 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.649654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:45701 (local address 127.0.0.1:46130)
0810 07:22:39.650000 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.650049 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:39.650169 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:39.651166 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:39.651170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:39.651645 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:39.651781 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.651787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.652938 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.652942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.653795 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.653806 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.654315 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.654318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.654440 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.654865 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:39.654876 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:39.656303 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:39.658335 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.658342 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.658347 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.658577 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.658867 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.658869 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.658870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.658943 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.659378 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:39.659383 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:39.659526 (+ 143us) client_negotiation.cc:769] Sending connection context
0810 07:22:39.660349 (+ 823us) client_negotiation.cc:240] Negotiation successful
0810 07:22:39.660557 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":99,"threads_started":1}
I20260810 07:22:39.660746 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.649732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46130 (local address 127.1.125.254:45701)
0810 07:22:39.649869 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.649874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.650136 (+ 262us) server_negotiation.cc:407] Connection header received
0810 07:22:39.650266 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.650269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.650322 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.650416 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.651916 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.652804 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.653932 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.654190 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.654302 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.656458 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.656462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.656464 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.656521 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.658229 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.658678 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.658681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.658683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.658774 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.659031 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.659034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.659036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.659247 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.659369 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.660442 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.660600 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 07:22:39.661792 22037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.252@KRBTEST.COM for kudu/127.1.125.253@KRBTEST.COM
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:39.670166 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.662297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:37293 (local address 127.0.0.1:55774)
0810 07:22:39.662408 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.662423 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:39.662494 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:39.662789 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:39.662792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:39.662959 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:39.663028 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.663033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.663802 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.663808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.664312 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.664319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.664863 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.664867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.664951 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.665276 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:39.665286 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:39.666629 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:39.668651 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.668656 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.668658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.668861 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.669159 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.669162 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.669164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.669254 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.669653 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:39.669658 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:39.669753 (+ 95us) client_negotiation.cc:769] Sending connection context
0810 07:22:39.669862 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 07:22:39.670001 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 07:22:39.670238 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.662431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55774 (local address 127.1.125.253:37293)
0810 07:22:39.662571 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.662576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.662601 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:22:39.662647 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.662649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.662686 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.662773 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.663158 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.663689 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.664442 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.664745 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.664861 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.666766 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.666770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.666772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.666801 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.668527 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.668966 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.668970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.668972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.669061 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.669349 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.669353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.669355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.669538 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.669650 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.669950 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.670068 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:22:39.670744 21812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } attempt: 3
I20260810 07:22:39.671156 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.661698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.1.125.254:45701)
0810 07:22:39.661809 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.661812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.661823 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:22:39.661953 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.661956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.662000 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.662075 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.662880 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.663357 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.664205 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.664426 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.664585 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.667373 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.667376 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.667379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.667410 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.669016 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.669429 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.669433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.669434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.669529 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.669809 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.669813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.669815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.669984 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.670213 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.670637 (+ 424us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.670988 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:22:39.671209 22037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:22:39.673923 21922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } attempt: 3
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.252@KRBTEST.COM for kudu/127.1.125.252@KRBTEST.COM
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for kudu/127.1.125.252@KRBTEST.COM
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.253@KRBTEST.COM
Aug 10 07:22:39 dist-test-slave-44d2 krb5kdc[21690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.253@KRBTEST.COM for kudu/127.1.125.252@KRBTEST.COM
I20260810 07:22:39.683107 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.671628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36164)
0810 07:22:39.671738 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.671752 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:39.671823 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:39.672693 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:39.672697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:39.672857 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:39.672966 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.672972 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.673758 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.673764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.674783 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.674792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.675380 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.675384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.675467 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.675976 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:39.675995 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:39.677873 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:39.681460 (+ 3587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.681464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.681466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.681715 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.682024 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.682028 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.682031 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.682135 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.682622 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:39.682625 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:39.682717 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 07:22:39.682780 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:22:39.682937 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":250}
I20260810 07:22:39.683436 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.671110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36154)
0810 07:22:39.671351 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.671363 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:39.671447 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:39.672287 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:39.672290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:39.672560 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:39.672720 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.672729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.674847 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.674851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.675748 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.675760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.676395 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.676399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.676536 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.677086 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:39.677100 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:39.678618 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:39.681145 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.681153 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.681157 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.681421 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.681727 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.681731 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.681733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.682098 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.682852 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:39.682857 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:39.682976 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 07:22:39.683073 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 07:22:39.683259 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174}
I20260810 07:22:39.683841 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.671802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36164 (local address 127.1.125.252:43497)
0810 07:22:39.672188 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.672195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.672206 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:22:39.672510 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.672514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.672564 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.672669 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.673098 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.673647 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.674894 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.675270 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.678016 (+ 2746us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.678128 (+ 112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.678132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.678135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.678152 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.681338 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.681826 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.681832 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.681834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.681914 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.682242 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.682247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.682250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.682512 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.683426 (+ 914us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.683511 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.683626 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":290,"spinlock_wait_cycles":2816,"thread_start_us":130,"threads_started":1}
I20260810 07:22:39.684053 22040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.671624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36154 (local address 127.1.125.252:43497)
0810 07:22:39.671966 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.671971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.671984 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:22:39.672016 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.672019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.672058 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.672258 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.673089 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.674723 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.675905 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.676269 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.676732 (+ 463us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.678777 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.678782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.678785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.678817 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.681021 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.681510 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.681516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.681518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.681603 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.682237 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.682243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.682246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.682486 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.683643 (+ 1157us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.683728 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.683837 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":269}
I20260810 07:22:39.686528 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.674293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:43497 (local address 127.0.0.1:36180)
0810 07:22:39.674548 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:39.674561 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:39.674647 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:39.675306 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:39.675312 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:39.675498 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:39.675624 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.675639 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.676921 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.676924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.677774 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:39.677784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.678973 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.678978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:39.679090 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.679568 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:39.679596 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:39.681215 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:39.684907 (+ 3692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.684910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.684912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.685178 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.685502 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:39.685506 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:39.685508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:39.685599 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:39.686034 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:39.686037 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:39.686135 (+ 98us) client_negotiation.cc:769] Sending connection context
0810 07:22:39.686250 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 07:22:39.686373 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181}
I20260810 07:22:39.686837 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.674367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36180 (local address 127.1.125.252:43497)
0810 07:22:39.675040 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.675048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.675090 (+ 42us) server_negotiation.cc:407] Connection header received
0810 07:22:39.675119 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.675122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.675159 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.675749 (+ 590us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.675894 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.676772 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.678348 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.678814 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.679294 (+ 480us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.681565 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.681571 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.681573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.681610 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.684797 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.685300 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.685303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.685305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.685390 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.685706 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.685709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.685711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.685915 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.686024 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.686336 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.686446 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":94,"threads_started":1}
I20260810 07:22:39.687394 21812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7: Bootstrap starting.
I20260810 07:22:39.687393 22037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23: Bootstrap starting.
I20260810 07:22:39.689659 21922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373: Bootstrap starting.
I20260810 07:22:39.689831 21812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:39.689898 22037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:39.690677 21812 log.cc:824] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:39.690761 22037 log.cc:824] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:39.691313 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.673037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55788 (local address 127.1.125.253:37293)
0810 07:22:39.673161 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.673166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.673181 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:22:39.673213 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.673215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.673257 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.674114 (+ 857us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.674221 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.674910 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.675946 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.676260 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.676732 (+ 472us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.680063 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.680066 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.680069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.680096 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.681959 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.682462 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.682466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.682468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.682550 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.682867 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.682870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.682872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.683108 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.683969 (+ 861us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.684061 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.684609 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:22:39.692031 21922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:39.692883 21922 log.cc:824] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:39.693184 21812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7: No bootstrap required, opened a new log
I20260810 07:22:39.693415 22037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23: No bootstrap required, opened a new log
I20260810 07:22:39.696154 21812 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } }
I20260810 07:22:39.696369 21812 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:39.696430 21812 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4cbf14818b1452b884b81a4345772c7, State: Initialized, Role: FOLLOWER
I20260810 07:22:39.696775 22037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } }
I20260810 07:22:39.697039 22037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:39.696967 21812 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } }
I20260810 07:22:39.697095 22037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 531ea8882bf04c0aa914b3730ffa7c23, State: Initialized, Role: FOLLOWER
I20260810 07:22:39.697527 22037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } }
I20260810 07:22:39.698274 21922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373: No bootstrap required, opened a new log
I20260810 07:22:39.698454 21812 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:39.698746 22037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:39.699314 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:22:39.702793 21922 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } }
I20260810 07:22:39.703047 21922 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:39.703104 21922 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1c595a55ce64ce5a7dce272e16e8373, State: Initialized, Role: FOLLOWER
I20260810 07:22:39.703536 21922 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } }
W20260810 07:22:39.704386 22086 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:22:39.704455 22086 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:22:39.704857 21922 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:39.704916 21708 master_runner.cc:186] Error getting master registration for 127.1.125.253:37293: OK, instance_id { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" instance_seqno: 1786346559417571 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:22:39.709518 21708 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:22:39.710984 22080 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 07:22:39.713191 21932 master_runner.cc:263] No leader master found from master 531ea8882bf04c0aa914b3730ffa7c23
I20260810 07:22:39.713259 21932 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:22:39.715358 21826 master_runner.cc:263] No leader master found from master a1c595a55ce64ce5a7dce272e16e8373
I20260810 07:22:39.715432 21826 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:22:39.716632 22080 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:22:39.717650 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.702760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46150 (local address 127.1.125.254:45701)
0810 07:22:39.703703 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.703707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.704348 (+ 641us) server_negotiation.cc:407] Connection header received
0810 07:22:39.704578 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.704583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.704635 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.704687 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.706064 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.706787 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.707972 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.708277 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.708382 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.713788 (+ 5406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.713792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.713795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.713823 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.715643 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.716177 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.716184 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.716185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.716285 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.716651 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.716655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.716657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.716874 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.716993 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.717361 (+ 368us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.717481 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":865}
W20260810 07:22:39.719401 22107 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:22:39.719503 22107 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:22:39.719774 22052 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } } }
I20260810 07:22:39.719192 22091 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } } }
I20260810 07:22:39.720558 22091 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:22:39.725030 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.706994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55802 (local address 127.1.125.253:37293)
0810 07:22:39.707105 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.707110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.711367 (+ 4257us) server_negotiation.cc:407] Connection header received
0810 07:22:39.711545 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.711550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.711615 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.711686 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.713661 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.714413 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.716851 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.717197 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.717278 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.721209 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.721214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.721219 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.721253 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.722963 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.723477 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.723481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.723484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.723579 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.723928 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.723931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.723933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.724168 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.724301 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.724736 (+ 435us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.724868 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:22:39.725462 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:39.703316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36194 (local address 127.1.125.252:43497)
0810 07:22:39.703447 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:39.703452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:39.709824 (+ 6372us) server_negotiation.cc:407] Connection header received
0810 07:22:39.710029 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:39.710034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:39.710083 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:39.710138 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:39.711130 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.711854 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.714843 (+ 2989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:39.715247 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:39.715334 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:39.720847 (+ 5513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:39.720851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:39.720854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:39.720884 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:39.723275 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.723778 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.723782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.723783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.723879 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:39.724205 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:39.724209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:39.724211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:39.724455 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:39.724603 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:39.725186 (+ 583us) server_negotiation.cc:299] Negotiation successful
0810 07:22:39.725314 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:22:39.726994 22052 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:22:39.722101 22055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } } }
I20260810 07:22:39.727244 22055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:22:39.859563 22052 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:39.859957 22052 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } }
I20260810 07:22:39.861964 21887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a4cbf14818b1452b884b81a4345772c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1c595a55ce64ce5a7dce272e16e8373" is_pre_election: true
I20260810 07:22:39.862317 21887 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4cbf14818b1452b884b81a4345772c7 in term 0.
I20260810 07:22:39.863082 21738 leader_election.cc:302] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1c595a55ce64ce5a7dce272e16e8373, a4cbf14818b1452b884b81a4345772c7; no voters:
I20260810 07:22:39.864411 21994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a4cbf14818b1452b884b81a4345772c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "531ea8882bf04c0aa914b3730ffa7c23" is_pre_election: true
I20260810 07:22:39.864745 21994 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4cbf14818b1452b884b81a4345772c7 in term 0.
I20260810 07:22:39.868234 22052 leader_election.cc:288] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1c595a55ce64ce5a7dce272e16e8373 (127.1.125.253:37293), 531ea8882bf04c0aa914b3730ffa7c23 (127.1.125.252:43497)
I20260810 07:22:39.868649 22052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:22:39.868794 22052 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:22:39.868898 22052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:39.870608 22052 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } }
I20260810 07:22:39.871340 21887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a4cbf14818b1452b884b81a4345772c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1c595a55ce64ce5a7dce272e16e8373"
I20260810 07:22:39.871506 21887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:39.873127 21887 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4cbf14818b1452b884b81a4345772c7 in term 1.
I20260810 07:22:39.873548 21738 leader_election.cc:302] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1c595a55ce64ce5a7dce272e16e8373, a4cbf14818b1452b884b81a4345772c7; no voters:
I20260810 07:22:39.874138 21994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a4cbf14818b1452b884b81a4345772c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "531ea8882bf04c0aa914b3730ffa7c23"
I20260810 07:22:39.874274 21994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:39.875429 21994 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4cbf14818b1452b884b81a4345772c7 in term 1.
I20260810 07:22:39.875821 22052 leader_election.cc:288] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [CANDIDATE]: Term 1 election: Requested vote from peers a1c595a55ce64ce5a7dce272e16e8373 (127.1.125.253:37293), 531ea8882bf04c0aa914b3730ffa7c23 (127.1.125.252:43497)
I20260810 07:22:39.876011 22052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:39.876127 22052 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [term 1 LEADER]: Becoming Leader. State: Replica: a4cbf14818b1452b884b81a4345772c7, State: Running, Role: LEADER
I20260810 07:22:39.876516 22052 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } }
I20260810 07:22:39.878425 22052 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4cbf14818b1452b884b81a4345772c7. Latest consensus state: current_term: 1 leader_uuid: "a4cbf14818b1452b884b81a4345772c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } } }
I20260810 07:22:39.878682 22052 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:39.879108 22115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:39.880105 22115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260810 07:22:39.882941 21962 tablet.cc:2369] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:22:39.885776 21887 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [term 1 FOLLOWER]: Refusing update from remote peer a4cbf14818b1452b884b81a4345772c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:39.885813 21994 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [term 1 FOLLOWER]: Refusing update from remote peer a4cbf14818b1452b884b81a4345772c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:39.886467 22111 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:39.887006 22111 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:39.890776 22055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4cbf14818b1452b884b81a4345772c7. Latest consensus state: current_term: 1 leader_uuid: "a4cbf14818b1452b884b81a4345772c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } } }
I20260810 07:22:39.890905 22055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:22:39.891253 22091 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4cbf14818b1452b884b81a4345772c7. Latest consensus state: current_term: 1 leader_uuid: "a4cbf14818b1452b884b81a4345772c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } } }
I20260810 07:22:39.891394 22091 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:22:39.895134 22111 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a4cbf14818b1452b884b81a4345772c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } } }
I20260810 07:22:39.895256 22111 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:39.896091 22052 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a4cbf14818b1452b884b81a4345772c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } } }
I20260810 07:22:39.896217 22052 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:39.896363 22055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a4cbf14818b1452b884b81a4345772c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } } }
I20260810 07:22:39.896507 22055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:22:39.896950 22117 mvcc.cc:205] Tried to move back new op lower bound from 7316875509283696640 to 7316875509257723904. Current Snapshot: MvccSnapshot[applied={T|T < 7316875509283696640}]
I20260810 07:22:39.897382 22091 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a4cbf14818b1452b884b81a4345772c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbf14818b1452b884b81a4345772c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45701 } } peers { permanent_uuid: "a1c595a55ce64ce5a7dce272e16e8373" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 37293 } } peers { permanent_uuid: "531ea8882bf04c0aa914b3730ffa7c23" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 43497 } } }
I20260810 07:22:39.897511 22091 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:22:39.899528 22115 catalog_manager.cc:1347] Generated new cluster ID: 948c47fbd960491e9fec64c58944d235
I20260810 07:22:39.899583 22115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:22:39.916980 22115 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:22:39.918380 22115 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:39.929738 22115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4cbf14818b1452b884b81a4345772c7: Generated new TSK 0
I20260810 07:22:39.930744 22115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:22:40.705905 22086 catalog_manager.cc:1259] Loaded cluster ID: 948c47fbd960491e9fec64c58944d235
I20260810 07:22:40.706024 22086 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23: loading cluster ID for follower catalog manager: success
I20260810 07:22:40.707928 22086 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23: acquiring CA information for follower catalog manager: success
I20260810 07:22:40.709142 22086 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 531ea8882bf04c0aa914b3730ffa7c23: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:22:40.720954 22107 catalog_manager.cc:1259] Loaded cluster ID: 948c47fbd960491e9fec64c58944d235
I20260810 07:22:40.721026 22107 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373: loading cluster ID for follower catalog manager: success
I20260810 07:22:40.722584 22107 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373: acquiring CA information for follower catalog manager: success
I20260810 07:22:40.723722 22107 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a1c595a55ce64ce5a7dce272e16e8373: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:22:40.997937 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 21708
I20260810 07:22:41.013206 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 21826
I20260810 07:22:41.025617 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 21932
2026-08-10T07:22:41Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3506 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:22:41 dist-test-slave-44d2 krb5kdc[22140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:22:41 dist-test-slave-44d2 krb5kdc[22140](info): set up 2 sockets
Aug 10 07:22:41 dist-test-slave-44d2 krb5kdc[22140](info): commencing operation
krb5kdc: starting...
W20260810 07:22:42.294565 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:22:42 dist-test-slave-44d2 krb5kdc[22140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:22:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:22:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:22:42.423830 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:37215
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:37215
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:42.539085 22163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:42.539356 22163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:42.539398 22163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:42.543460 22163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:42.543522 22163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:42.543542 22163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:42.543608 22163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:42.543640 22163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:42.548895 22163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:37215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:37215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:42.549971 22163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:42.551069 22163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:42.560137 22173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:42.560148 22178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:42.560101 22170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:42.560362 22163 server_base.cc:1034] running on GCE node
I20260810 07:22:42.561004 22163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:42.562108 22163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:42.563306 22163 hybrid_clock.cc:648] HybridClock initialized: now 1786346562563289 us; error 32 us; skew 500 ppm
Aug 10 07:22:42 dist-test-slave-44d2 krb5kdc[22140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346562, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:42.566092 22163 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:22:42.567063 22163 webserver.cc:460] Webserver started at http://127.1.125.254:33539/ using document root <none> and password file <none>
I20260810 07:22:42.567655 22163 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:42.567754 22163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:42.567998 22163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:42.570139 22163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "17a8bcf9e9c94bdd827a738e83bc9524"
format_stamp: "Formatted at 2026-08-10 07:22:42 on dist-test-slave-44d2"
server_key: "ad615cebf4f9abcf5f25c2b253ad73c1"
server_key_iv: "8d46e3d227e83d8b0f7f7ffc894f24ce"
server_key_version: "encryptionkey@0"
I20260810 07:22:42.570731 22163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "17a8bcf9e9c94bdd827a738e83bc9524"
format_stamp: "Formatted at 2026-08-10 07:22:42 on dist-test-slave-44d2"
server_key: "ad615cebf4f9abcf5f25c2b253ad73c1"
server_key_iv: "8d46e3d227e83d8b0f7f7ffc894f24ce"
server_key_version: "encryptionkey@0"
I20260810 07:22:42.574627 22163 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:22:42.577180 22187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:42.578382 22163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:22:42.578531 22163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "17a8bcf9e9c94bdd827a738e83bc9524"
format_stamp: "Formatted at 2026-08-10 07:22:42 on dist-test-slave-44d2"
server_key: "ad615cebf4f9abcf5f25c2b253ad73c1"
server_key_iv: "8d46e3d227e83d8b0f7f7ffc894f24ce"
server_key_version: "encryptionkey@0"
I20260810 07:22:42.578656 22163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:42.593916 22163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:42.596824 22163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:42.597033 22163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:42.604744 22163 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:37215
I20260810 07:22:42.604776 22268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:37215 every 8 connection(s)
I20260810 07:22:42.605721 22163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:42.608599 22269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:42.610814 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 22163
I20260810 07:22:42.610917 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:22:42.611215 1527 external_mini_cluster.cc:1428] Setting key 874b76c1ded381e5750fe898798759eb
I20260810 07:22:42.614565 22269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524: Bootstrap starting.
I20260810 07:22:42.616900 22269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:42.617625 22269 log.cc:824] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:42.620076 22269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524: No bootstrap required, opened a new log
Aug 10 07:22:42 dist-test-slave-44d2 krb5kdc[22140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:42.624473 22269 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a8bcf9e9c94bdd827a738e83bc9524" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 37215 } }
I20260810 07:22:42.624773 22269 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:42.624853 22269 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a8bcf9e9c94bdd827a738e83bc9524, State: Initialized, Role: FOLLOWER
I20260810 07:22:42.625351 22269 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a8bcf9e9c94bdd827a738e83bc9524" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 37215 } }
I20260810 07:22:42.625528 22269 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:42.625603 22269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:42.625701 22269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:42.626839 22269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a8bcf9e9c94bdd827a738e83bc9524" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 37215 } }
I20260810 07:22:42.627198 22269 leader_election.cc:302] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17a8bcf9e9c94bdd827a738e83bc9524; no voters:
I20260810 07:22:42.627526 22269 leader_election.cc:288] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:22:42.627642 22276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:42.627707 22273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:42.612673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55690 (local address 127.1.125.254:37215)
0810 07:22:42.613122 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:42.613138 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:42.613175 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:22:42.613818 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:42.613835 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:42.614176 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:42.614519 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:42.615394 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:42.616450 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:42.617471 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:42.618091 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:42.618270 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:42.621165 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:42.621172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:42.621189 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:42.621222 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:42.624010 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:42.624840 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:42.624846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:42.624852 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:42.625013 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:42.625367 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:42.625373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:42.625375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:42.625920 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:42.626128 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:42.626391 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 07:22:42.626710 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":124,"threads_started":1}
I20260810 07:22:42.627912 22276 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [term 1 LEADER]: Becoming Leader. State: Replica: 17a8bcf9e9c94bdd827a738e83bc9524, State: Running, Role: LEADER
I20260810 07:22:42.628743 22276 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a8bcf9e9c94bdd827a738e83bc9524" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 37215 } }
I20260810 07:22:42.628871 22269 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:42.630532 22278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17a8bcf9e9c94bdd827a738e83bc9524" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a8bcf9e9c94bdd827a738e83bc9524" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 37215 } } }
I20260810 07:22:42.630678 22278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:42.631655 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:22:42.631125 22279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17a8bcf9e9c94bdd827a738e83bc9524. Latest consensus state: current_term: 1 leader_uuid: "17a8bcf9e9c94bdd827a738e83bc9524" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a8bcf9e9c94bdd827a738e83bc9524" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 37215 } } }
I20260810 07:22:42.632437 22279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524 [sys.catalog]: This master's current role is: LEADER
W20260810 07:22:42.634613 22300 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:22:42.634707 22300 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:22:42.634786 22289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:42.635475 22289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:42.640717 22289 catalog_manager.cc:1347] Generated new cluster ID: ab939ce10e054044a0b6508b3de19052
I20260810 07:22:42.640790 22289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:22:42.650240 22289 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:22:42.651456 22289 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:42.661284 22289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17a8bcf9e9c94bdd827a738e83bc9524: Generated new TSK 0
I20260810 07:22:42.661952 22289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:22:42.688830 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 22163
2026-08-10T07:22:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1665 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:22:42 dist-test-slave-44d2 krb5kdc[22307](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:22:42 dist-test-slave-44d2 krb5kdc[22307](info): set up 2 sockets
Aug 10 07:22:42 dist-test-slave-44d2 krb5kdc[22307](info): commencing operation
krb5kdc: starting...
W20260810 07:22:43.971653 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:22:44 dist-test-slave-44d2 krb5kdc[22307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:22:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:22:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:22:44.103631 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38631
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:38631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:44.221861 22322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:44.222167 22322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:44.222240 22322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:44.226330 22322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:44.226416 22322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:44.226451 22322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:44.226491 22322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:44.226526 22322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:44.231734 22322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:38631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:44.232937 22322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:44.233997 22322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:44.242621 22334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:44.242697 22331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:44.242667 22322 server_base.cc:1034] running on GCE node
W20260810 07:22:44.242769 22330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:44.243490 22322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:44.244622 22322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:44.245815 22322 hybrid_clock.cc:648] HybridClock initialized: now 1786346564245796 us; error 30 us; skew 500 ppm
Aug 10 07:22:44 dist-test-slave-44d2 krb5kdc[22307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346564, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:44.248629 22322 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:22:44.249468 22322 webserver.cc:460] Webserver started at http://127.1.125.254:46759/ using document root <none> and password file <none>
I20260810 07:22:44.250051 22322 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:44.250113 22322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:44.250275 22322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:44.252233 22322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "7b98ed3a20a54625baa78c3364cb116c"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "b4bf7a3723f5e54ece93a42848032538"
server_key_iv: "46488786a67e21e6942942e185d23ca4"
server_key_version: "encryptionkey@0"
I20260810 07:22:44.252813 22322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "7b98ed3a20a54625baa78c3364cb116c"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "b4bf7a3723f5e54ece93a42848032538"
server_key_iv: "46488786a67e21e6942942e185d23ca4"
server_key_version: "encryptionkey@0"
I20260810 07:22:44.256398 22322 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:22:44.258960 22345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:44.260205 22322 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:22:44.260304 22322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "7b98ed3a20a54625baa78c3364cb116c"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "b4bf7a3723f5e54ece93a42848032538"
server_key_iv: "46488786a67e21e6942942e185d23ca4"
server_key_version: "encryptionkey@0"
I20260810 07:22:44.260381 22322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:44.276086 22322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:44.279392 22322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:44.279553 22322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:44.287230 22322 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:38631
I20260810 07:22:44.287540 22418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:38631 every 8 connection(s)
I20260810 07:22:44.288304 22322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:44.290355 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 22322
I20260810 07:22:44.290467 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:22:44.290787 1527 external_mini_cluster.cc:1428] Setting key 9e95501d09dfcf64e4b98e0262290f12
I20260810 07:22:44.291391 22422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:44.297343 22422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Bootstrap starting.
Aug 10 07:22:44 dist-test-slave-44d2 krb5kdc[22307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:44.299799 22422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:44.300717 22422 log.cc:824] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:44.302757 22422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: No bootstrap required, opened a new log
I20260810 07:22:44.304797 22425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:44.292209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.1.125.254:38631)
0810 07:22:44.292704 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:44.292719 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:44.292754 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:22:44.293398 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:44.293414 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:44.293732 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:44.294061 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:44.294896 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:44.295875 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:44.296813 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:44.297285 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:44.297464 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:44.299953 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:44.299960 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:44.299979 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:44.300011 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:44.301925 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:44.302383 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:44.302388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:44.302393 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:44.302507 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:44.302803 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:44.302807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:44.302809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:44.303247 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:44.303458 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:44.303734 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 07:22:44.304019 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":180,"threads_started":1}
I20260810 07:22:44.306521 22422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } }
I20260810 07:22:44.306769 22422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:44.306861 22422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b98ed3a20a54625baa78c3364cb116c, State: Initialized, Role: FOLLOWER
I20260810 07:22:44.307384 22422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } }
I20260810 07:22:44.307548 22422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:44.307642 22422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:44.307804 22422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:44.308827 22422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } }
I20260810 07:22:44.309175 22422 leader_election.cc:302] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b98ed3a20a54625baa78c3364cb116c; no voters:
I20260810 07:22:44.309499 22422 leader_election.cc:288] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:22:44.309754 22429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:44.310001 22429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 1 LEADER]: Becoming Leader. State: Replica: 7b98ed3a20a54625baa78c3364cb116c, State: Running, Role: LEADER
I20260810 07:22:44.310289 22429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } }
I20260810 07:22:44.310806 22422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:44.311978 22435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b98ed3a20a54625baa78c3364cb116c. Latest consensus state: current_term: 1 leader_uuid: "7b98ed3a20a54625baa78c3364cb116c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } } }
I20260810 07:22:44.312116 22435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:44.312536 22445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:44.312436 22430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b98ed3a20a54625baa78c3364cb116c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } } }
I20260810 07:22:44.312668 22430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:44.315557 22445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:44.321668 22445 catalog_manager.cc:1347] Generated new cluster ID: 668254fc929340d3b1d307aab67312fb
I20260810 07:22:44.321738 22445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:22:44.329901 22445 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:22:44.331071 22445 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:44.346881 22445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Generated new TSK 0
I20260810 07:22:44.347685 22445 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:22:44.380259 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38631
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:44.495121 22466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:44.495368 22466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:44.495409 22466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:44.499450 22466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:44.499526 22466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:44.499584 22466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:22:44.504724 22466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:44.505750 22466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:44.506808 22466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:44.516557 22478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:44.516616 22474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:44.516557 22476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:44.516741 22466 server_base.cc:1034] running on GCE node
I20260810 07:22:44.517215 22466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:44.517892 22466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:44.519083 22466 hybrid_clock.cc:648] HybridClock initialized: now 1786346564519066 us; error 28 us; skew 500 ppm
Aug 10 07:22:44 dist-test-slave-44d2 krb5kdc[22307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346564, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:44.522320 22466 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:22:44.523285 22466 webserver.cc:460] Webserver started at http://127.1.125.193:41447/ using document root <none> and password file <none>
I20260810 07:22:44.523886 22466 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:44.523958 22466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:44.524216 22466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:44.526458 22466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "989c791077be44fb912e71852395a9c8"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "b83cef3e6a6ca8b1f01c7f373f821a23"
server_key_iv: "5230b56facf8a0e854844ee86c999553"
server_key_version: "encryptionkey@0"
I20260810 07:22:44.527084 22466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "989c791077be44fb912e71852395a9c8"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "b83cef3e6a6ca8b1f01c7f373f821a23"
server_key_iv: "5230b56facf8a0e854844ee86c999553"
server_key_version: "encryptionkey@0"
I20260810 07:22:44.531193 22466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 07:22:44.533722 22490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:44.534880 22466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:22:44.535023 22466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "989c791077be44fb912e71852395a9c8"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "b83cef3e6a6ca8b1f01c7f373f821a23"
server_key_iv: "5230b56facf8a0e854844ee86c999553"
server_key_version: "encryptionkey@0"
I20260810 07:22:44.535135 22466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:44.554713 22466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:44.557761 22466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:44.557969 22466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:44.558588 22466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:44.559655 22466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:44.559751 22466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:44.559823 22466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:44.559870 22466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:44.569475 22466 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:32849
I20260810 07:22:44.569509 22664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:32849 every 8 connection(s)
I20260810 07:22:44.570364 22466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:22:44.577379 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 22466
I20260810 07:22:44.577535 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:22:44.577862 1527 external_mini_cluster.cc:1428] Setting key 9216c5144046829bda36551d15a83009
Aug 10 07:22:44 dist-test-slave-44d2 krb5kdc[22307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346564, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:44.584946 22425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:44.572159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:52335 (local address 127.1.125.254:38631)
0810 07:22:44.572293 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:44.572305 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:44.573221 (+ 916us) server_negotiation.cc:407] Connection header received
0810 07:22:44.574468 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:44.574471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:44.574521 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:44.574606 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:44.575739 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:44.576262 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:44.577470 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:44.577842 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:44.577963 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:44.580807 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:44.580816 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:44.580817 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:44.580841 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:44.582605 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:44.583157 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:44.583162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:44.583164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:44.583271 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:44.583626 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:44.583633 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:44.583637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:44.583901 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:44.584014 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:44.584537 (+ 523us) server_negotiation.cc:299] Negotiation successful
0810 07:22:44.584696 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:22:44.585763 22667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:44.572546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:52335)
0810 07:22:44.573062 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:44.573105 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:44.574269 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:44.574766 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:44.574772 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:44.575144 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:44.575607 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:44.575620 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:44.576370 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:44.576373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:44.577337 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:44.577347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:44.578024 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:44.578029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:44.578250 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:44.578905 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:44.578922 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:44.580645 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:44.582736 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:44.582741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:44.582754 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:44.583022 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:44.583374 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:44.583377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:44.583378 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:44.583518 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:44.584011 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:44.584016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:44.584248 (+ 232us) client_negotiation.cc:769] Sending connection context
0810 07:22:44.584399 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 07:22:44.584748 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":125,"threads_started":1}
I20260810 07:22:44.587116 22665 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38631
I20260810 07:22:44.587358 22665 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:44.587867 22665 heartbeater.cc:511] Master 127.1.125.254:38631 requested a full tablet report, sending...
I20260810 07:22:44.589437 22365 ts_manager.cc:194] Registered new tserver with Master: 989c791077be44fb912e71852395a9c8 (127.1.125.193:32849)
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:22:44.590620 22365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:52335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:22:44.611377 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38631
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:44.729957 22676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:44.730203 22676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:44.730247 22676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:44.734138 22676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:44.734193 22676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:44.734251 22676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:22:44.739413 22676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:44.740355 22676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:44.741330 22676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:22:44.751073 22676 server_base.cc:1034] running on GCE node
W20260810 07:22:44.751104 22682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:44.751124 22683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:44.751124 22686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:44.753726 22676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:44.754354 22676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:44.755554 22676 hybrid_clock.cc:648] HybridClock initialized: now 1786346564755539 us; error 26 us; skew 500 ppm
Aug 10 07:22:44 dist-test-slave-44d2 krb5kdc[22307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346564, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:44.758333 22676 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:22:44.759240 22676 webserver.cc:460] Webserver started at http://127.1.125.194:40391/ using document root <none> and password file <none>
I20260810 07:22:44.759804 22676 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:44.759895 22676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:44.760134 22676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:44.762207 22676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "ec8b94c361a342bea5ee5ae6ece95433"
server_key_iv: "ee93e6f5d848468269281871d5915e81"
server_key_version: "encryptionkey@0"
I20260810 07:22:44.762832 22676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "ec8b94c361a342bea5ee5ae6ece95433"
server_key_iv: "ee93e6f5d848468269281871d5915e81"
server_key_version: "encryptionkey@0"
I20260810 07:22:44.766631 22676 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:22:44.769014 22696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:44.770185 22676 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:22:44.770323 22676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "ec8b94c361a342bea5ee5ae6ece95433"
server_key_iv: "ee93e6f5d848468269281871d5915e81"
server_key_version: "encryptionkey@0"
I20260810 07:22:44.770431 22676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:44.790318 22676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:44.793277 22676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:44.793504 22676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:44.794103 22676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:44.794951 22676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:44.795024 22676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:44.795100 22676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:44.795152 22676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:44.804930 22676 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:42571
I20260810 07:22:44.804975 22867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:42571 every 8 connection(s)
I20260810 07:22:44.805810 22676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:22:44.809027 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 22676
I20260810 07:22:44.809183 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:22:44.809540 1527 external_mini_cluster.cc:1428] Setting key c6a1bee94b8968948fc470ccc6c37e19
Aug 10 07:22:44 dist-test-slave-44d2 krb5kdc[22307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346564, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:44.821766 22425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:44.807551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:52283 (local address 127.1.125.254:38631)
0810 07:22:44.807715 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:44.807720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:44.808548 (+ 828us) server_negotiation.cc:407] Connection header received
0810 07:22:44.810339 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:44.810342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:44.810400 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:44.810484 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:44.812226 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:44.812870 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:44.814355 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:44.814612 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:44.814735 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:44.817535 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:44.817542 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:44.817547 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:44.817595 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:44.819393 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:44.819934 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:44.819939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:44.819944 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:44.820052 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:44.820391 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:44.820395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:44.820399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:44.820708 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:44.820828 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:44.821376 (+ 548us) server_negotiation.cc:299] Negotiation successful
0810 07:22:44.821526 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
I20260810 07:22:44.822526 22870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:44.807936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:52283)
0810 07:22:44.808379 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:44.808422 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:44.810051 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:44.810675 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:44.810682 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:44.811296 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:44.812051 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:44.812074 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:44.813047 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:44.813054 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:44.814217 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:44.814228 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:44.814780 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:44.814784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:44.814967 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:44.815590 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:44.815607 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:44.817359 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:44.819524 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:44.819529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:44.819541 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:44.819794 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:44.820156 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:44.820158 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:44.820159 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:44.820290 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:44.820819 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:44.820824 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:44.821054 (+ 230us) client_negotiation.cc:769] Sending connection context
0810 07:22:44.821279 (+ 225us) client_negotiation.cc:240] Negotiation successful
0810 07:22:44.821565 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":114,"threads_started":1}
I20260810 07:22:44.823845 22868 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38631
I20260810 07:22:44.824075 22868 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:44.824682 22868 heartbeater.cc:511] Master 127.1.125.254:38631 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
I20260810 07:22:44.825937 22365 ts_manager.cc:194] Registered new tserver with Master: 4ffc59b0e3a24f2cae42442c9d86f4f6 (127.1.125.194:42571)
I20260810 07:22:44.826534 22365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:52283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:22:44.841756 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38631
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:44.961653 22875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:44.961952 22875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:44.962059 22875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:44.966377 22875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:44.966449 22875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:44.966581 22875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:22:44.971873 22875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:44.972991 22875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:44.974061 22875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:22:44.984123 22875 server_base.cc:1034] running on GCE node
W20260810 07:22:44.984217 22881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:44.984157 22883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:44.984313 22880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:44.984776 22875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:44.985460 22875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:44.986632 22875 hybrid_clock.cc:648] HybridClock initialized: now 1786346564986615 us; error 27 us; skew 500 ppm
Aug 10 07:22:44 dist-test-slave-44d2 krb5kdc[22307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346564, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:44.989449 22875 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:22:44.990428 22875 webserver.cc:460] Webserver started at http://127.1.125.195:37381/ using document root <none> and password file <none>
I20260810 07:22:44.991001 22875 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:44.991122 22875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:44.991369 22875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:22:44.993557 22875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "580b02fcc6e74a12814b4f8e0ee51d3b"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "dc735303668e334a1c46e8b80a7710e9"
server_key_iv: "ebcf4d1164674128a4f5d9a5ed975911"
server_key_version: "encryptionkey@0"
I20260810 07:22:44.994153 22875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "580b02fcc6e74a12814b4f8e0ee51d3b"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "dc735303668e334a1c46e8b80a7710e9"
server_key_iv: "ebcf4d1164674128a4f5d9a5ed975911"
server_key_version: "encryptionkey@0"
I20260810 07:22:44.997932 22875 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:22:45.000399 22892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:45.001654 22875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:22:45.001806 22875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "580b02fcc6e74a12814b4f8e0ee51d3b"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "dc735303668e334a1c46e8b80a7710e9"
server_key_iv: "ebcf4d1164674128a4f5d9a5ed975911"
server_key_version: "encryptionkey@0"
I20260810 07:22:45.001927 22875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:45.019848 22875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:45.022773 22875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:45.022992 22875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:45.023628 22875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:45.024519 22875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:45.024595 22875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:45.024662 22875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:45.024706 22875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:45.035228 22875 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:43925
I20260810 07:22:45.035244 23063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:43925 every 8 connection(s)
I20260810 07:22:45.036168 22875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:22:45.038681 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 22875
I20260810 07:22:45.038848 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:22:45.039149 1527 external_mini_cluster.cc:1428] Setting key f65979294ca41960366cc292205d3ac3
Aug 10 07:22:45 dist-test-slave-44d2 krb5kdc[22307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346564, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:45.050895 22425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.038271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:51879 (local address 127.1.125.254:38631)
0810 07:22:45.038432 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:45.038439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:45.039600 (+ 1161us) server_negotiation.cc:407] Connection header received
0810 07:22:45.041279 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:45.041283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:45.041345 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:45.041428 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:45.042750 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.043347 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.044787 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.045052 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.045182 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.047682 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:45.047686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:45.047688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:45.047715 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:45.049134 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:45.049597 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:45.049600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:45.049601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:45.049676 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:45.049968 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:45.049970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:45.049971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:45.050153 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:45.050231 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:45.050679 (+ 448us) server_negotiation.cc:299] Negotiation successful
0810 07:22:45.050777 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:22:45.051756 23067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.038795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:51879)
0810 07:22:45.039419 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:45.039479 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:45.041030 (+ 1551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:45.041645 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:45.041653 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:45.042062 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:45.042587 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.042603 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.043501 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.043504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.044648 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.044660 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.045226 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.045231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.045429 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.045974 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:45.045991 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:45.047518 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:45.049241 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:45.049245 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:45.049257 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:45.049508 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:45.049754 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:45.049756 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:45.049757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:45.049885 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:45.050243 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:45.050248 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:45.050478 (+ 230us) client_negotiation.cc:769] Sending connection context
0810 07:22:45.050632 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 07:22:45.050865 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":172,"threads_started":1}
I20260810 07:22:45.052971 23064 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38631
I20260810 07:22:45.053236 23064 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:45.053704 23064 heartbeater.cc:511] Master 127.1.125.254:38631 requested a full tablet report, sending...
I20260810 07:22:45.054703 22365 ts_manager.cc:194] Registered new tserver with Master: 580b02fcc6e74a12814b4f8e0ee51d3b (127.1.125.195:43925)
I20260810 07:22:45.055260 22365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:51879
I20260810 07:22:45.065057 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:22:45.074218 22425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.067035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54988 (local address 127.1.125.254:38631)
0810 07:22:45.067189 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:45.067194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:45.067340 (+ 146us) server_negotiation.cc:407] Connection header received
0810 07:22:45.067438 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:45.067440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:45.067480 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:45.067544 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:45.068227 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.068794 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.069599 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.069805 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.069879 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.071228 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:45.071231 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:45.071233 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:45.071249 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:45.072583 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:45.072979 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:45.072982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:45.072983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:45.073059 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:45.073296 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:45.073298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:45.073299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:45.073565 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:45.073675 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:45.073909 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 07:22:45.073999 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 07:22:45.076978 22365 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:22:45.080200 22365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:22:45.094856 23077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.088864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:32849 (local address 127.0.0.1:38960)
0810 07:22:45.089608 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:45.089664 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:45.089874 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:45.090662 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:45.090668 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:45.090692 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:45.090820 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.090828 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.092005 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.092012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.093152 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.093163 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.094362 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.094364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.094440 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.094465 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 07:22:45.094590 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 07:22:45.094711 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":1,"thread_start_us":78,"threads_started":1}
I20260810 07:22:45.094866 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.089019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.1.125.193:32849)
0810 07:22:45.089354 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:45.089361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:45.089760 (+ 399us) server_negotiation.cc:407] Connection header received
0810 07:22:45.090072 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:45.090078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:45.090301 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:45.090433 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:45.090957 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.091880 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.093298 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.094112 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.094173 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.094284 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:45.094625 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 07:22:45.094720 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":128,"threads_started":1}
I20260810 07:22:45.095412 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.089053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:42571 (local address 127.0.0.1:37440)
0810 07:22:45.089919 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:45.089933 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:45.090013 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:45.090976 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:45.090979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:45.090995 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:45.091068 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.091073 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.092705 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.092709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.093939 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.093950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.095003 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.095007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.095109 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.095134 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 07:22:45.095190 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 07:22:45.095254 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"mutex_wait_us":70,"thread_start_us":108,"threads_started":1}
I20260810 07:22:45.095620 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.089591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:43925 (local address 127.0.0.1:42754)
0810 07:22:45.090360 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:45.090373 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:45.090470 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:45.091161 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:45.091164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:45.091184 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:45.091273 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.091279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.092757 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.092761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.093831 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.093842 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.095185 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.095189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.095265 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.095284 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:22:45.095348 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:22:45.095402 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":27,"thread_start_us":105,"threads_started":1}
I20260810 07:22:45.095850 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.089792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42754 (local address 127.1.125.195:43925)
0810 07:22:45.090280 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:45.090292 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:45.090586 (+ 294us) server_negotiation.cc:407] Connection header received
0810 07:22:45.090678 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:45.090682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:45.090831 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:45.090990 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:45.091417 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.092606 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.093966 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.094849 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.094975 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.095118 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:45.095518 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 07:22:45.095659 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":148,"threads_started":1}
I20260810 07:22:45.096700 23084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.089586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37440 (local address 127.1.125.194:42571)
0810 07:22:45.090502 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:45.090519 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:45.090543 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:22:45.090626 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:45.090631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:45.090818 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:45.091198 (+ 380us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:45.091325 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.092549 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.094081 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.094864 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.096224 (+ 1360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.096383 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:45.096433 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 07:22:45.096546 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":814,"thread_start_us":124,"threads_started":1}
I20260810 07:22:45.098632 22967 tablet_service.cc:1467] Processing CreateTablet for tablet 20f7baa350344fcd80c7ace28cc19ea0 (DEFAULT_TABLE table=test-table [id=f4096c5752de4cc78662e3dff2faeee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:45.098650 22559 tablet_service.cc:1467] Processing CreateTablet for tablet 20f7baa350344fcd80c7ace28cc19ea0 (DEFAULT_TABLE table=test-table [id=f4096c5752de4cc78662e3dff2faeee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:45.099663 22967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20f7baa350344fcd80c7ace28cc19ea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:45.099321 22762 tablet_service.cc:1467] Processing CreateTablet for tablet 20f7baa350344fcd80c7ace28cc19ea0 (DEFAULT_TABLE table=test-table [id=f4096c5752de4cc78662e3dff2faeee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:45.099742 22559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20f7baa350344fcd80c7ace28cc19ea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:45.100287 22762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20f7baa350344fcd80c7ace28cc19ea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:45.105963 23088 tablet_bootstrap.cc:492] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b: Bootstrap starting.
I20260810 07:22:45.107511 23089 tablet_bootstrap.cc:492] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Bootstrap starting.
I20260810 07:22:45.108119 23091 tablet_bootstrap.cc:492] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6: Bootstrap starting.
I20260810 07:22:45.108474 23088 tablet_bootstrap.cc:654] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:45.109588 23088 log.cc:824] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:45.109748 23089 tablet_bootstrap.cc:654] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:45.110723 23091 tablet_bootstrap.cc:654] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:45.110692 23089 log.cc:824] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:45.111636 23091 log.cc:824] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:45.112293 23088 tablet_bootstrap.cc:492] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b: No bootstrap required, opened a new log
I20260810 07:22:45.112530 23088 ts_tablet_manager.cc:1397] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260810 07:22:45.112816 23089 tablet_bootstrap.cc:492] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: No bootstrap required, opened a new log
I20260810 07:22:45.113008 23089 ts_tablet_manager.cc:1397] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:22:45.113763 23091 tablet_bootstrap.cc:492] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6: No bootstrap required, opened a new log
I20260810 07:22:45.113961 23091 ts_tablet_manager.cc:1397] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:22:45.115801 23088 raft_consensus.cc:348] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } } peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } }
I20260810 07:22:45.116055 23088 raft_consensus.cc:374] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:45.116132 23088 raft_consensus.cc:729] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580b02fcc6e74a12814b4f8e0ee51d3b, State: Initialized, Role: FOLLOWER
I20260810 07:22:45.116650 23088 consensus_queue.cc:260] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } } peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } }
I20260810 07:22:45.117359 23064 heartbeater.cc:503] Master 127.1.125.254:38631 was elected leader, sending a full tablet report...
I20260810 07:22:45.117590 23089 raft_consensus.cc:348] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } } peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } }
I20260810 07:22:45.117830 23089 raft_consensus.cc:374] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:45.117887 23089 raft_consensus.cc:729] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989c791077be44fb912e71852395a9c8, State: Initialized, Role: FOLLOWER
I20260810 07:22:45.117915 23091 raft_consensus.cc:348] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } } peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } }
I20260810 07:22:45.118238 23091 raft_consensus.cc:374] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:45.118294 23091 raft_consensus.cc:729] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ffc59b0e3a24f2cae42442c9d86f4f6, State: Initialized, Role: FOLLOWER
I20260810 07:22:45.118371 23089 consensus_queue.cc:260] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } } peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } }
I20260810 07:22:45.119022 22665 heartbeater.cc:503] Master 127.1.125.254:38631 was elected leader, sending a full tablet report...
I20260810 07:22:45.119360 23089 ts_tablet_manager.cc:1428] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:22:45.118793 23091 consensus_queue.cc:260] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } } peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } }
I20260810 07:22:45.120443 22868 heartbeater.cc:503] Master 127.1.125.254:38631 was elected leader, sending a full tablet report...
I20260810 07:22:45.120743 23091 ts_tablet_manager.cc:1428] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260810 07:22:45.120995 23088 ts_tablet_manager.cc:1428] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260810 07:22:45.174851 23101 raft_consensus.cc:482] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:45.175140 23101 raft_consensus.cc:504] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } } peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } }
I20260810 07:22:45.176348 23101 leader_election.cc:288] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ffc59b0e3a24f2cae42442c9d86f4f6 (127.1.125.194:42571), 580b02fcc6e74a12814b4f8e0ee51d3b (127.1.125.195:43925)
I20260810 07:22:45.180670 23084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.176703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:59125 (local address 127.1.125.194:42571)
0810 07:22:45.176864 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:45.176869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:45.177133 (+ 264us) server_negotiation.cc:407] Connection header received
0810 07:22:45.177353 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:45.177359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:45.177420 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:45.177500 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:45.177883 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.178391 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.179638 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.180078 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.180213 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.180271 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:45.180463 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 07:22:45.180562 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 07:22:45.180696 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.176671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:42571 (local address 127.1.125.193:59125)
0810 07:22:45.177030 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:45.177042 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:45.177196 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:45.177536 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:45.177541 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:45.177562 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:45.177755 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.177762 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.178516 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.178522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.179501 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.179512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.180208 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.180212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.180326 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.180345 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:22:45.180430 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 07:22:45.180525 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":32,"thread_start_us":150,"threads_started":1}
I20260810 07:22:45.180920 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.176722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:54805 (local address 127.1.125.195:43925)
0810 07:22:45.176876 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:45.176881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:45.177276 (+ 395us) server_negotiation.cc:407] Connection header received
0810 07:22:45.177419 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:45.177421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:45.177489 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:45.177571 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:45.177920 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.178419 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.179647 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.180179 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.180297 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.180345 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:45.180694 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 07:22:45.180774 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 07:22:45.181010 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.176722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:43925 (local address 127.1.125.193:54805)
0810 07:22:45.177171 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:45.177184 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:45.177309 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:45.177610 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:45.177614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:45.177638 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:45.177768 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.177772 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.178566 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.178569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.179505 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.179515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.180408 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.180412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.180539 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.180565 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:22:45.180675 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 07:22:45.180745 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":63,"thread_start_us":114,"threads_started":1}
I20260810 07:22:45.181459 22794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f7baa350344fcd80c7ace28cc19ea0" candidate_uuid: "989c791077be44fb912e71852395a9c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" is_pre_election: true
I20260810 07:22:45.181632 22996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f7baa350344fcd80c7ace28cc19ea0" candidate_uuid: "989c791077be44fb912e71852395a9c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" is_pre_election: true
I20260810 07:22:45.181798 22794 raft_consensus.cc:2454] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 989c791077be44fb912e71852395a9c8 in term 0.
I20260810 07:22:45.181957 22996 raft_consensus.cc:2454] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 989c791077be44fb912e71852395a9c8 in term 0.
I20260810 07:22:45.182327 22492 leader_election.cc:302] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ffc59b0e3a24f2cae42442c9d86f4f6, 989c791077be44fb912e71852395a9c8; no voters:
I20260810 07:22:45.182629 23101 raft_consensus.cc:2781] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:22:45.182721 23101 raft_consensus.cc:482] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:22:45.182765 23101 raft_consensus.cc:3037] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:45.184005 23101 raft_consensus.cc:504] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } } peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } }
I20260810 07:22:45.184425 23101 leader_election.cc:288] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [CANDIDATE]: Term 1 election: Requested vote from peers 4ffc59b0e3a24f2cae42442c9d86f4f6 (127.1.125.194:42571), 580b02fcc6e74a12814b4f8e0ee51d3b (127.1.125.195:43925)
I20260810 07:22:45.184794 22794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f7baa350344fcd80c7ace28cc19ea0" candidate_uuid: "989c791077be44fb912e71852395a9c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6"
I20260810 07:22:45.184851 22996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f7baa350344fcd80c7ace28cc19ea0" candidate_uuid: "989c791077be44fb912e71852395a9c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b"
I20260810 07:22:45.184947 22794 raft_consensus.cc:3037] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:45.184980 22996 raft_consensus.cc:3037] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:45.186018 22794 raft_consensus.cc:2454] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 989c791077be44fb912e71852395a9c8 in term 1.
I20260810 07:22:45.186079 22996 raft_consensus.cc:2454] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 989c791077be44fb912e71852395a9c8 in term 1.
I20260810 07:22:45.186331 22492 leader_election.cc:302] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ffc59b0e3a24f2cae42442c9d86f4f6, 989c791077be44fb912e71852395a9c8; no voters:
I20260810 07:22:45.186501 23101 raft_consensus.cc:2781] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:45.186715 23101 raft_consensus.cc:686] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 1 LEADER]: Becoming Leader. State: Replica: 989c791077be44fb912e71852395a9c8, State: Running, Role: LEADER
I20260810 07:22:45.187075 23101 consensus_queue.cc:237] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } } peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } }
I20260810 07:22:45.190289 22364 catalog_manager.cc:5495] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 989c791077be44fb912e71852395a9c8 (127.1.125.193). New cstate: current_term: 1 leader_uuid: "989c791077be44fb912e71852395a9c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } health_report { overall_health: UNKNOWN } } }
I20260810 07:22:45.216627 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.212823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38966 (local address 127.1.125.193:32849)
0810 07:22:45.212985 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:45.212989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:45.213005 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:22:45.213190 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:45.213193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:45.213251 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:45.213366 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:22:45.213630 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.214179 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.215081 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.215314 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.215428 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.215758 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:22:45.216181 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:22:45.216321 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:45.216387 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 07:22:45.216445 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:22:45.222570 22996 raft_consensus.cc:1261] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b [term 1 FOLLOWER]: Refusing update from remote peer 989c791077be44fb912e71852395a9c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:45.222573 22794 raft_consensus.cc:1261] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 1 FOLLOWER]: Refusing update from remote peer 989c791077be44fb912e71852395a9c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:45.223438 23106 consensus_queue.cc:1035] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:45.223663 23101 consensus_queue.cc:1035] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:45.231930 23109 mvcc.cc:205] Tried to move back new op lower bound from 7316875531143983104 to 7316875531008958464. Current Snapshot: MvccSnapshot[applied={T|T < 7316875531143983104}]
I20260810 07:22:45.243608 22364 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:45.243757 22364 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:45.246307 22364 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Sending DeleteTablet for 3 replicas of tablet 20f7baa350344fcd80c7ace28cc19ea0
I20260810 07:22:45.247193 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 22466
I20260810 07:22:45.247186 22967 tablet_service.cc:1514] Processing DeleteTablet for tablet 20f7baa350344fcd80c7ace28cc19ea0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:45 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:42754
I20260810 07:22:45.247258 22762 tablet_service.cc:1514] Processing DeleteTablet for tablet 20f7baa350344fcd80c7ace28cc19ea0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:45 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:37440
I20260810 07:22:45.256840 23127 tablet_replica.cc:330] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6: stopping tablet replica
I20260810 07:22:45.257233 23127 raft_consensus.cc:2229] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:45.257602 23127 raft_consensus.cc:2258] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:45.258111 23126 tablet_replica.cc:330] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b: stopping tablet replica
I20260810 07:22:45.258468 23126 raft_consensus.cc:2229] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:45.258788 23126 raft_consensus.cc:2258] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:45.259722 23127 ts_tablet_manager.cc:1905] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:45.260157 23126 ts_tablet_manager.cc:1905] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:45.262416 23126 ts_tablet_manager.cc:1918] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:45.262544 23126 log.cc:1196] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/20f7baa350344fcd80c7ace28cc19ea0
I20260810 07:22:45.262866 23126 ts_tablet_manager.cc:1939] T 20f7baa350344fcd80c7ace28cc19ea0 P 580b02fcc6e74a12814b4f8e0ee51d3b: Deleting consensus metadata
I20260810 07:22:45.263386 23127 ts_tablet_manager.cc:1918] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:45.263566 23127 log.cc:1196] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/20f7baa350344fcd80c7ace28cc19ea0
I20260810 07:22:45.264045 23127 ts_tablet_manager.cc:1939] T 20f7baa350344fcd80c7ace28cc19ea0 P 4ffc59b0e3a24f2cae42442c9d86f4f6: Deleting consensus metadata
I20260810 07:22:45.264303 22350 catalog_manager.cc:4841] TS 580b02fcc6e74a12814b4f8e0ee51d3b (127.1.125.195:43925): tablet 20f7baa350344fcd80c7ace28cc19ea0 (table test-table [id=f4096c5752de4cc78662e3dff2faeee1]) successfully deleted
W20260810 07:22:45.265203 22350 connection.cc:535] client connection to 127.1.125.193:32849 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:22:45.265384 22350 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:22:45.265528 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 22676
I20260810 07:22:45.269892 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.269413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:32849 (local address 127.0.0.1:38982)
0810 07:22:45.269658 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:45.269763 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:32849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260810 07:22:45.270716 22350 catalog_manager.cc:4584] TS 989c791077be44fb912e71852395a9c8 (127.1.125.193:32849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20f7baa350344fcd80c7ace28cc19ea0: Network error: Client connection negotiation failed: client connection to 127.1.125.193:32849: connect: Connection refused (error 111)
I20260810 07:22:45.278144 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 22875
I20260810 07:22:45.282578 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.278331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:42571 (local address 127.0.0.1:37444)
0810 07:22:45.282374 (+ 4043us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:45.282459 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.194:42571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3923}
I20260810 07:22:45.290441 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 22322
I20260810 07:22:45.304230 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38631
--webserver_interface=127.1.125.254
--webserver_port=46759
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:38631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:45.417951 23129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:45.418212 23129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:45.418293 23129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:45.422199 23129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:45.422254 23129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:45.422273 23129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:45.422331 23129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:45.422363 23129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:45.427497 23129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:38631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=46759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:45.428601 23129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:45.429685 23129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:45.444659 23141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:45.444654 23137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:45.444653 23138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:45.445114 23129 server_base.cc:1034] running on GCE node
I20260810 07:22:45.445705 23129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:45.446863 23129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:45.448055 23129 hybrid_clock.cc:648] HybridClock initialized: now 1786346565448031 us; error 35 us; skew 500 ppm
Aug 10 07:22:45 dist-test-slave-44d2 krb5kdc[22307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:45.450927 23129 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:22:45.451776 23129 webserver.cc:460] Webserver started at http://127.1.125.254:46759/ using document root <none> and password file <none>
I20260810 07:22:45.452320 23129 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:45.452394 23129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:45.456218 23129 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260810 07:22:45.458362 23150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:45.459581 23129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:22:45.459684 23129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "7b98ed3a20a54625baa78c3364cb116c"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "b4bf7a3723f5e54ece93a42848032538"
server_key_iv: "46488786a67e21e6942942e185d23ca4"
server_key_version: "encryptionkey@0"
I20260810 07:22:45.460249 23129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:45.478960 23129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:45.482450 23129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:45.482623 23129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:45.490025 23129 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:38631
I20260810 07:22:45.490335 23247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:38631 every 8 connection(s)
I20260810 07:22:45.491147 23129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:45.491524 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 23129
I20260810 07:22:45.492038 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:32849
--local_ip_for_outbound_sockets=127.1.125.193
--tserver_master_addrs=127.1.125.254:38631
--webserver_port=41447
--webserver_interface=127.1.125.193
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:45.495767 23248 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:22:45.497512 23248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Bootstrap starting.
I20260810 07:22:45.515348 23248 log.cc:824] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:45.524191 23248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:45.524641 23248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Bootstrap complete.
I20260810 07:22:45.528861 23248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } }
I20260810 07:22:45.529291 23248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b98ed3a20a54625baa78c3364cb116c, State: Initialized, Role: FOLLOWER
I20260810 07:22:45.529835 23248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } }
I20260810 07:22:45.530002 23248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:45.530098 23248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:45.530217 23248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:22:45.532091 23248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } }
I20260810 07:22:45.532433 23248 leader_election.cc:302] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b98ed3a20a54625baa78c3364cb116c; no voters:
I20260810 07:22:45.532869 23248 leader_election.cc:288] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:22:45.533011 23254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:22:45.533322 23254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 2 LEADER]: Becoming Leader. State: Replica: 7b98ed3a20a54625baa78c3364cb116c, State: Running, Role: LEADER
I20260810 07:22:45.533653 23254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } }
I20260810 07:22:45.534145 23248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:45.534657 23257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b98ed3a20a54625baa78c3364cb116c. Latest consensus state: current_term: 2 leader_uuid: "7b98ed3a20a54625baa78c3364cb116c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } } }
I20260810 07:22:45.534785 23257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:45.535069 23256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7b98ed3a20a54625baa78c3364cb116c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } } }
I20260810 07:22:45.535148 23259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:45.535179 23256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:45.540362 23259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:45.541005 23259 catalog_manager.cc:1259] Loaded cluster ID: 668254fc929340d3b1d307aab67312fb
I20260810 07:22:45.541069 23259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:22:45.542469 23259 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 07:22:45.546948 23259 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 07:22:45.552456 23259 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:22:45.553246 23259 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Loaded TSK: 0
I20260810 07:22:45.554461 23259 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:22:45.616380 23252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:45.616710 23252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:45.616780 23252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:45.620999 23252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:45.621059 23252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:45.621146 23252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:22:45.626513 23252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:32849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=41447
--tserver_master_addrs=127.1.125.254:38631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:45.627581 23252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:45.628715 23252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:45.639848 23287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:45.639850 23289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:45.640005 23252 server_base.cc:1034] running on GCE node
W20260810 07:22:45.639897 23286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:45.640604 23252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:45.641366 23252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:45.642577 23252 hybrid_clock.cc:648] HybridClock initialized: now 1786346565642553 us; error 43 us; skew 500 ppm
Aug 10 07:22:45 dist-test-slave-44d2 krb5kdc[22307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:45.645578 23252 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:22:45.646549 23252 webserver.cc:460] Webserver started at http://127.1.125.193:41447/ using document root <none> and password file <none>
I20260810 07:22:45.647135 23252 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:45.647210 23252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:45.651365 23252 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:22:45.653764 23300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:45.655200 23252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 07:22:45.655339 23252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "989c791077be44fb912e71852395a9c8"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "b83cef3e6a6ca8b1f01c7f373f821a23"
server_key_iv: "5230b56facf8a0e854844ee86c999553"
server_key_version: "encryptionkey@0"
I20260810 07:22:45.655890 23252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:45.675233 23252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:45.678792 23252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:45.679020 23252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:45.679744 23252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:45.681103 23308 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:22:45.685475 23252 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:22:45.685542 23252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260810 07:22:45.685592 23252 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:22:45.687340 23252 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:22:45.687402 23252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260810 07:22:45.687690 23308 tablet_bootstrap.cc:492] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Bootstrap starting.
I20260810 07:22:45.698961 23467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:32849 every 8 connection(s)
I20260810 07:22:45.698958 23252 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:32849
I20260810 07:22:45.700357 23252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:22:45.710103 23308 log.cc:824] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:45.710482 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 23252
I20260810 07:22:45.711021 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:42571
--local_ip_for_outbound_sockets=127.1.125.194
--tserver_master_addrs=127.1.125.254:38631
--webserver_port=40391
--webserver_interface=127.1.125.194
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:22:45 dist-test-slave-44d2 krb5kdc[22307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:45.719333 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.702734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:59823)
0810 07:22:45.703293 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:45.703336 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:45.704201 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:45.705438 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:45.705446 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:45.705897 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:45.706361 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.706375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.707737 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.707740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.708689 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.708701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.709431 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.709435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.709635 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.710330 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:45.710361 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:45.712039 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:45.715382 (+ 3343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:45.715391 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:45.715408 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:45.715826 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:45.716357 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:45.716364 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:45.716369 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:45.716626 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:45.717548 (+ 922us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:45.717554 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:45.717831 (+ 277us) client_negotiation.cc:769] Sending connection context
0810 07:22:45.718036 (+ 205us) client_negotiation.cc:240] Negotiation successful
0810 07:22:45.718372 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":169,"threads_started":1}
I20260810 07:22:45.719415 23308 tablet_bootstrap.cc:492] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:45.719858 23308 tablet_bootstrap.cc:492] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Bootstrap complete.
I20260810 07:22:45.720072 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.702681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:59823 (local address 127.1.125.254:38631)
0810 07:22:45.703081 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:45.703105 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:45.703471 (+ 366us) server_negotiation.cc:407] Connection header received
0810 07:22:45.704691 (+ 1220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:45.704703 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:45.705031 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:45.705381 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:45.706554 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.707555 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.708832 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.709297 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.709477 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.712246 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:45.712251 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:45.712263 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:45.712291 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:45.715219 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:45.716049 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:45.716056 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:45.716062 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:45.716231 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:45.716790 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:45.716796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:45.716801 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:45.717383 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:45.717550 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:45.718222 (+ 672us) server_negotiation.cc:299] Negotiation successful
0810 07:22:45.718618 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":131,"threads_started":1}
I20260810 07:22:45.721364 23308 ts_tablet_manager.cc:1397] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Time spent bootstrapping tablet: real 0.034s user 0.021s sys 0.011s
I20260810 07:22:45.722698 23468 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38631
I20260810 07:22:45.722993 23468 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:45.723661 23468 heartbeater.cc:511] Master 127.1.125.254:38631 requested a full tablet report, sending...
I20260810 07:22:45.724117 23308 raft_consensus.cc:348] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } } peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } }
I20260810 07:22:45.724627 23308 raft_consensus.cc:729] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989c791077be44fb912e71852395a9c8, State: Initialized, Role: FOLLOWER
I20260810 07:22:45.725135 23308 consensus_queue.cc:260] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } } peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } }
I20260810 07:22:45.726178 23308 ts_tablet_manager.cc:1428] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:22:45.727218 23183 ts_manager.cc:194] Registered new tserver with Master: 989c791077be44fb912e71852395a9c8 (127.1.125.193:32849)
I20260810 07:22:45.731324 23183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:59823
I20260810 07:22:45.734819 23468 heartbeater.cc:503] Master 127.1.125.254:38631 was elected leader, sending a full tablet report...
Aug 10 07:22:45 dist-test-slave-44d2 krb5kdc[22307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for kudu/127.1.125.193@KRBTEST.COM
I20260810 07:22:45.742815 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.730742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:32849 (local address 127.0.0.1:38988)
0810 07:22:45.731498 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:45.731554 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:45.731726 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:45.732287 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:45.732292 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:45.732824 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:45.732958 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.732968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.734458 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.734462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.735494 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.735505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.736331 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.736335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.736438 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.737126 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:45.737139 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:45.738786 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:45.740974 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:45.740984 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:45.740989 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:45.741216 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:45.741618 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:45.741620 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:45.741622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:45.741704 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:45.742227 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:45.742231 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:45.742335 (+ 104us) client_negotiation.cc:769] Sending connection context
0810 07:22:45.742449 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 07:22:45.742679 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":128,"threads_started":1}
I20260810 07:22:45.742933 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.730789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38988 (local address 127.1.125.193:32849)
0810 07:22:45.731209 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:45.731221 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:45.731651 (+ 430us) server_negotiation.cc:407] Connection header received
0810 07:22:45.731886 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:45.731891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:45.732049 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:45.732216 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:45.733103 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.734293 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.735625 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.736208 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.736313 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.738942 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:45.738955 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:45.738972 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:45.739001 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:45.740850 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:45.741377 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:45.741387 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:45.741392 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:45.741507 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:45.741794 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:45.741798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:45.741799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:45.742116 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:45.742249 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:45.742562 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 07:22:45.742755 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":102,"threads_started":1}
I20260810 07:22:45.743984 23373 tablet_service.cc:1514] Processing DeleteTablet for tablet 20f7baa350344fcd80c7ace28cc19ea0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:45 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:38988
I20260810 07:22:45.744911 23482 tablet_replica.cc:330] stopping tablet replica
I20260810 07:22:45.745251 23482 raft_consensus.cc:2229] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:45.745512 23482 raft_consensus.cc:2258] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:45.746681 23482 ts_tablet_manager.cc:1905] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:45.751463 23482 ts_tablet_manager.cc:1918] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:45.751575 23482 log.cc:1196] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal/wals/20f7baa350344fcd80c7ace28cc19ea0
I20260810 07:22:45.751883 23482 ts_tablet_manager.cc:1939] T 20f7baa350344fcd80c7ace28cc19ea0 P 989c791077be44fb912e71852395a9c8: Deleting consensus metadata
I20260810 07:22:45.754074 23154 catalog_manager.cc:4841] TS 989c791077be44fb912e71852395a9c8 (127.1.125.193:32849): tablet 20f7baa350344fcd80c7ace28cc19ea0 (table test-table [id=f4096c5752de4cc78662e3dff2faeee1]) successfully deleted
W20260810 07:22:45.841516 23475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:45.841758 23475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:45.841799 23475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:45.845896 23475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:45.845953 23475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:45.846010 23475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:22:45.851302 23475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:42571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=40391
--tserver_master_addrs=127.1.125.254:38631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:45.852305 23475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:45.853394 23475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:45.863415 23491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:45.863373 23490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:45.863507 23475 server_base.cc:1034] running on GCE node
W20260810 07:22:45.863377 23495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:45.864005 23475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:45.864827 23475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:45.866101 23475 hybrid_clock.cc:648] HybridClock initialized: now 1786346565866067 us; error 31 us; skew 500 ppm
Aug 10 07:22:45 dist-test-slave-44d2 krb5kdc[22307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:45.869143 23475 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:22:45.870102 23475 webserver.cc:460] Webserver started at http://127.1.125.194:40391/ using document root <none> and password file <none>
I20260810 07:22:45.870705 23475 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:45.870826 23475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:45.874886 23475 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:22:45.877055 23505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:45.878398 23475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:22:45.878574 23475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "ec8b94c361a342bea5ee5ae6ece95433"
server_key_iv: "ee93e6f5d848468269281871d5915e81"
server_key_version: "encryptionkey@0"
I20260810 07:22:45.879180 23475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:45.894541 23475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:45.898236 23475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:45.898483 23475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:45.899149 23475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:45.900087 23475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:45.900167 23475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:45.900247 23475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:45.900295 23475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:45.910338 23475 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:42571
I20260810 07:22:45.910384 23685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:42571 every 8 connection(s)
I20260810 07:22:45.911448 23475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:22:45.920668 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 23475
I20260810 07:22:45.921159 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:43925
--local_ip_for_outbound_sockets=127.1.125.195
--tserver_master_addrs=127.1.125.254:38631
--webserver_port=37381
--webserver_interface=127.1.125.195
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:22:45 dist-test-slave-44d2 krb5kdc[22307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:45.927100 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.913496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:57077 (local address 127.1.125.254:38631)
0810 07:22:45.913663 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:45.913667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:45.914598 (+ 931us) server_negotiation.cc:407] Connection header received
0810 07:22:45.915926 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:45.915929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:45.915982 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:45.916058 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:45.917231 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.917759 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.919180 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.919405 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.919481 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.922364 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:45.922367 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:45.922369 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:45.922393 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:45.924221 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:45.925143 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:45.925149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:45.925154 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:45.925251 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:45.925646 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:45.925652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:45.925657 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:45.925927 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:45.926049 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:45.926628 (+ 579us) server_negotiation.cc:299] Negotiation successful
0810 07:22:45.926819 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:22:45.927796 23688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:45.913921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:57077)
0810 07:22:45.914429 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:45.914474 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:45.915720 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:45.916200 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:45.916207 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:45.916613 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:45.917102 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.917115 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.917911 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.917916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.919060 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:45.919072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:45.919555 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:45.919559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:45.919777 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:45.920581 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:45.920597 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:45.922180 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:45.924359 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:45.924364 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:45.924377 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:45.925014 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:45.925363 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:45.925365 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:45.925366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:45.925518 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:45.926058 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:45.926063 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:45.926320 (+ 257us) client_negotiation.cc:769] Sending connection context
0810 07:22:45.926509 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 07:22:45.926794 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":122,"threads_started":1}
I20260810 07:22:45.929365 23686 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38631
I20260810 07:22:45.929674 23686 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:45.930368 23686 heartbeater.cc:511] Master 127.1.125.254:38631 requested a full tablet report, sending...
I20260810 07:22:45.931635 23183 ts_manager.cc:194] Registered new tserver with Master: 4ffc59b0e3a24f2cae42442c9d86f4f6 (127.1.125.194:42571)
I20260810 07:22:45.932238 23183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:57077
W20260810 07:22:46.040375 23689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:46.040732 23689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:46.040838 23689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:46.044858 23689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:46.044930 23689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:46.045025 23689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:22:46.050333 23689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:43925
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=37381
--tserver_master_addrs=127.1.125.254:38631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:46.051365 23689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:46.052775 23689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:46.063308 23700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:46.063314 23697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:46.063400 23689 server_base.cc:1034] running on GCE node
W20260810 07:22:46.063292 23696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:46.063984 23689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:46.064805 23689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:46.065980 23689 hybrid_clock.cc:648] HybridClock initialized: now 1786346566065952 us; error 44 us; skew 500 ppm
Aug 10 07:22:46 dist-test-slave-44d2 krb5kdc[22307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:46.068922 23689 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:22:46.069805 23689 webserver.cc:460] Webserver started at http://127.1.125.195:37381/ using document root <none> and password file <none>
I20260810 07:22:46.070379 23689 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:46.070461 23689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:46.074329 23689 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:22:46.076341 23712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:46.077622 23689 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:22:46.077720 23689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "580b02fcc6e74a12814b4f8e0ee51d3b"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "dc735303668e334a1c46e8b80a7710e9"
server_key_iv: "ebcf4d1164674128a4f5d9a5ed975911"
server_key_version: "encryptionkey@0"
I20260810 07:22:46.078238 23689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:46.095959 23689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:46.098824 23689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:46.098984 23689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:46.099530 23689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:46.100310 23689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:46.100358 23689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:46.100397 23689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:46.100417 23689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:46.109961 23689 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:43925
I20260810 07:22:46.110005 23890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:43925 every 8 connection(s)
I20260810 07:22:46.110970 23689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:22:46 dist-test-slave-44d2 krb5kdc[22307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:22:46.120689 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 23689
I20260810 07:22:46.124955 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.112864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:48485 (local address 127.1.125.254:38631)
0810 07:22:46.113008 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:46.113012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:46.113871 (+ 859us) server_negotiation.cc:407] Connection header received
0810 07:22:46.115220 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:46.115224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:46.115285 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:46.115393 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:46.116614 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.117146 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.118299 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.118532 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.118647 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:46.121178 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:46.121183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:46.121186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:46.121221 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:46.122840 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:46.123364 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:46.123367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:46.123369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:46.123453 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:46.123818 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:46.123823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:46.123825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:46.124005 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:46.124114 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:46.124654 (+ 540us) server_negotiation.cc:299] Negotiation successful
0810 07:22:46.124780 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:22:46.125877 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.113267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:48485)
0810 07:22:46.113714 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:46.113758 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:46.115000 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:46.115590 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:46.115597 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:46.115951 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:46.116419 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:46.116433 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.117279 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.117283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:46.118164 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:46.118174 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.118689 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.118693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:46.118865 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:46.119406 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:46.119422 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:46.121015 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:46.122966 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:46.122972 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:46.122995 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:46.123256 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:46.123565 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:46.123568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:46.123570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:46.123720 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:46.124115 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:46.124122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:46.124361 (+ 239us) client_negotiation.cc:769] Sending connection context
0810 07:22:46.124605 (+ 244us) client_negotiation.cc:240] Negotiation successful
0810 07:22:46.124920 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260810 07:22:46.127050 23894 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38631
I20260810 07:22:46.127311 23894 heartbeater.cc:464] Registering TS with master...
I20260810 07:22:46.127808 23894 heartbeater.cc:511] Master 127.1.125.254:38631 requested a full tablet report, sending...
I20260810 07:22:46.128990 23183 ts_manager.cc:194] Registered new tserver with Master: 580b02fcc6e74a12814b4f8e0ee51d3b (127.1.125.195:43925)
I20260810 07:22:46.129715 23183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:48485
I20260810 07:22:46.130048 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.121420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55000 (local address 127.1.125.254:38631)
0810 07:22:46.121710 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:46.121715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:46.121936 (+ 221us) server_negotiation.cc:407] Connection header received
0810 07:22:46.122123 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:46.122125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:46.122165 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:46.122261 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:46.123013 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.123516 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.124405 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.124730 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.124837 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:46.126669 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:46.126675 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:46.126679 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:46.126707 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:46.128295 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:46.128829 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:46.128832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:46.128833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:46.128908 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:46.129206 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:46.129208 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:46.129210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:46.129363 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:46.129510 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:46.129709 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 07:22:46.129830 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":107,"threads_started":1}
I20260810 07:22:46.131804 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:22:46.136960 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.132424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55012 (local address 127.1.125.254:38631)
0810 07:22:46.132608 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:46.132613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:46.132771 (+ 158us) server_negotiation.cc:407] Connection header received
0810 07:22:46.132934 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:46.132937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:46.133028 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:46.133144 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:22:46.133369 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.133871 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.135074 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.135467 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.135576 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:46.135923 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:22:46.136445 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:22:46.136617 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:46.136746 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 07:22:46.136828 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:22:46.137826 23183 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:55012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:22:46.141095 23183 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:22:46.149482 23373 tablet_service.cc:1467] Processing CreateTablet for tablet 05e9dc6291894611a6449c6d447aca8d (DEFAULT_TABLE table=test-table [id=8eaec630c8844e15bfae66cf670d9387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:46.150233 23373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e9dc6291894611a6449c6d447aca8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:46.153502 23308 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8: Bootstrap starting.
I20260810 07:22:46.154464 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.148449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:43925 (local address 127.0.0.1:42762)
0810 07:22:46.148946 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:46.148960 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:46.149040 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:46.150046 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:46.150049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:46.150068 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:46.150171 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:46.150177 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.151651 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.151654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:46.152752 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:46.152762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.153964 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.153969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:46.154082 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:46.154130 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 07:22:46.154215 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 07:22:46.154286 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":143,"threads_started":1}
I20260810 07:22:46.154515 23308 tablet_bootstrap.cc:654] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:46.154774 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.148671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42762 (local address 127.1.125.195:43925)
0810 07:22:46.149159 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:46.149171 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:46.149194 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:22:46.149270 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:46.149275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:46.149638 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:46.149852 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:46.150415 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.151496 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.152895 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.153835 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.153946 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:46.154086 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:46.154275 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 07:22:46.154389 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":176,"threads_started":1}
I20260810 07:22:46.156150 23308 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8: No bootstrap required, opened a new log
I20260810 07:22:46.156258 23308 ts_tablet_manager.cc:1397] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260810 07:22:46.156777 23308 raft_consensus.cc:348] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:46.156908 23308 raft_consensus.cc:374] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:46.156949 23308 raft_consensus.cc:729] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989c791077be44fb912e71852395a9c8, State: Initialized, Role: FOLLOWER
I20260810 07:22:46.157049 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.147769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:42571 (local address 127.0.0.1:37458)
0810 07:22:46.147902 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:46.147918 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:46.147999 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:46.152553 (+ 4554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:46.152556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:46.152574 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:46.152672 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:46.152678 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.154358 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.154362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:46.155455 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:46.155494 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.156603 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.156606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:46.156706 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:46.156729 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:22:46.156830 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 07:22:46.156910 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 07:22:46.157087 23308 consensus_queue.cc:260] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:46.157366 23308 ts_tablet_manager.cc:1428] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:22:46.157934 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.147950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37458 (local address 127.1.125.194:42571)
0810 07:22:46.148290 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:46.148305 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:46.148330 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:22:46.148419 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:46.148425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:46.148696 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:46.148886 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:46.152908 (+ 4022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.154160 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.155658 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.156442 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.156606 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:46.156748 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:46.156866 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 07:22:46.156961 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":142,"threads_started":1}
I20260810 07:22:46.158128 23782 tablet_service.cc:1467] Processing CreateTablet for tablet 05e9dc6291894611a6449c6d447aca8d (DEFAULT_TABLE table=test-table [id=8eaec630c8844e15bfae66cf670d9387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:46.159333 23782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e9dc6291894611a6449c6d447aca8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:46.160780 23585 tablet_service.cc:1467] Processing CreateTablet for tablet 05e9dc6291894611a6449c6d447aca8d (DEFAULT_TABLE table=test-table [id=8eaec630c8844e15bfae66cf670d9387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:22:46.161842 23585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e9dc6291894611a6449c6d447aca8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:46.167896 23915 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Bootstrap starting.
I20260810 07:22:46.169730 23916 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: Bootstrap starting.
I20260810 07:22:46.170426 23915 tablet_bootstrap.cc:654] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:46.171208 23915 log.cc:824] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:46.172176 23916 tablet_bootstrap.cc:654] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: Neither blocks nor log segments found. Creating new log.
I20260810 07:22:46.172992 23916 log.cc:824] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:46.173619 23915 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: No bootstrap required, opened a new log
I20260810 07:22:46.173838 23915 ts_tablet_manager.cc:1397] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:22:46.174810 23916 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: No bootstrap required, opened a new log
I20260810 07:22:46.175025 23916 ts_tablet_manager.cc:1397] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:22:46.176836 23915 raft_consensus.cc:348] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:46.177064 23915 raft_consensus.cc:374] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:46.177124 23915 raft_consensus.cc:729] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580b02fcc6e74a12814b4f8e0ee51d3b, State: Initialized, Role: FOLLOWER
I20260810 07:22:46.177591 23915 consensus_queue.cc:260] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:46.178268 23894 heartbeater.cc:503] Master 127.1.125.254:38631 was elected leader, sending a full tablet report...
I20260810 07:22:46.178160 23916 raft_consensus.cc:348] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:46.178450 23916 raft_consensus.cc:374] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:22:46.178514 23916 raft_consensus.cc:729] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ffc59b0e3a24f2cae42442c9d86f4f6, State: Initialized, Role: FOLLOWER
I20260810 07:22:46.178696 23915 ts_tablet_manager.cc:1428] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:22:46.178965 23916 consensus_queue.cc:260] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:46.179677 23686 heartbeater.cc:503] Master 127.1.125.254:38631 was elected leader, sending a full tablet report...
I20260810 07:22:46.179949 23916 ts_tablet_manager.cc:1428] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260810 07:22:46.201799 23470 tablet.cc:2369] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:22:46.322983 23922 raft_consensus.cc:482] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:46.323267 23922 raft_consensus.cc:504] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:46.324689 23922 leader_election.cc:288] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 580b02fcc6e74a12814b4f8e0ee51d3b (127.1.125.195:43925), 989c791077be44fb912e71852395a9c8 (127.1.125.193:32849)
I20260810 07:22:46.329037 23688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.324998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:43925 (local address 127.1.125.194:54333)
0810 07:22:46.325150 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:46.325169 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:46.325280 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:46.325704 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:46.325707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:46.325733 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:46.325868 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:46.325877 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.326816 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.326820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:46.327739 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:46.327750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.328547 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.328551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:46.328682 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:46.328710 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 07:22:46.328795 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 07:22:46.328869 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 07:22:46.329082 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.325196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:54333 (local address 127.1.125.195:43925)
0810 07:22:46.325348 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:46.325353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:46.325372 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:22:46.325453 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:46.325458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:46.325513 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:46.325630 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:46.326021 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.326696 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.327883 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.328409 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.328559 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:46.328605 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:46.328819 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 07:22:46.328946 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 07:22:46.329557 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.325324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:32849 (local address 127.1.125.194:38453)
0810 07:22:46.325764 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:46.325786 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:46.325862 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:46.326160 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:46.326163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:46.326182 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:22:46.326307 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:46.326313 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.327191 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.327194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:46.328095 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:46.328106 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.328968 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.328972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:46.329071 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:46.329241 (+ 170us) client_negotiation.cc:769] Sending connection context
0810 07:22:46.329303 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 07:22:46.329410 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":164,"threads_started":1}
I20260810 07:22:46.329753 23815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e9dc6291894611a6449c6d447aca8d" candidate_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" is_pre_election: true
I20260810 07:22:46.329938 23925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.325562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:38453 (local address 127.1.125.193:32849)
0810 07:22:46.325853 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:46.325857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:46.325872 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:22:46.325932 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:46.325937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:46.326024 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:46.326122 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:22:46.326439 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.327053 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.328232 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.328843 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.329528 (+ 685us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:46.329691 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:46.329730 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 07:22:46.329797 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":88,"threads_started":1}
I20260810 07:22:46.330121 23815 raft_consensus.cc:2454] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ffc59b0e3a24f2cae42442c9d86f4f6 in term 0.
I20260810 07:22:46.330413 23407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e9dc6291894611a6449c6d447aca8d" candidate_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "989c791077be44fb912e71852395a9c8" is_pre_election: true
I20260810 07:22:46.330711 23407 raft_consensus.cc:2454] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ffc59b0e3a24f2cae42442c9d86f4f6 in term 0.
I20260810 07:22:46.330699 23509 leader_election.cc:302] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ffc59b0e3a24f2cae42442c9d86f4f6, 580b02fcc6e74a12814b4f8e0ee51d3b; no voters:
I20260810 07:22:46.331012 23922 raft_consensus.cc:2781] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:22:46.331125 23922 raft_consensus.cc:482] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:22:46.331164 23922 raft_consensus.cc:3037] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:46.332320 23922 raft_consensus.cc:504] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:46.332728 23922 leader_election.cc:288] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [CANDIDATE]: Term 1 election: Requested vote from peers 580b02fcc6e74a12814b4f8e0ee51d3b (127.1.125.195:43925), 989c791077be44fb912e71852395a9c8 (127.1.125.193:32849)
I20260810 07:22:46.333169 23815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e9dc6291894611a6449c6d447aca8d" candidate_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b"
I20260810 07:22:46.333169 23407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e9dc6291894611a6449c6d447aca8d" candidate_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "989c791077be44fb912e71852395a9c8"
I20260810 07:22:46.333298 23407 raft_consensus.cc:3037] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:46.333298 23815 raft_consensus.cc:3037] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:22:46.334611 23815 raft_consensus.cc:2454] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ffc59b0e3a24f2cae42442c9d86f4f6 in term 1.
I20260810 07:22:46.334616 23407 raft_consensus.cc:2454] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ffc59b0e3a24f2cae42442c9d86f4f6 in term 1.
I20260810 07:22:46.334892 23509 leader_election.cc:302] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ffc59b0e3a24f2cae42442c9d86f4f6, 989c791077be44fb912e71852395a9c8; no voters:
I20260810 07:22:46.335074 23922 raft_consensus.cc:2781] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:22:46.335393 23922 raft_consensus.cc:686] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 1 LEADER]: Becoming Leader. State: Replica: 4ffc59b0e3a24f2cae42442c9d86f4f6, State: Running, Role: LEADER
I20260810 07:22:46.335762 23922 consensus_queue.cc:237] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:46.339203 23181 catalog_manager.cc:5495] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ffc59b0e3a24f2cae42442c9d86f4f6 (127.1.125.194). New cstate: current_term: 1 leader_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } health_report { overall_health: UNKNOWN } } }
W20260810 07:22:46.361982 23895 tablet.cc:2369] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:22:46.392540 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.389119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37462 (local address 127.1.125.194:42571)
0810 07:22:46.389262 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:46.389266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:46.389319 (+ 53us) server_negotiation.cc:407] Connection header received
0810 07:22:46.389471 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:46.389474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:46.389554 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:46.389653 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:22:46.389867 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.390360 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.391158 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:46.391366 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:46.391440 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:46.391709 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:22:46.392151 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:22:46.392239 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:46.392304 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 07:22:46.392354 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:22:46.398862 23815 raft_consensus.cc:1261] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [term 1 FOLLOWER]: Refusing update from remote peer 4ffc59b0e3a24f2cae42442c9d86f4f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:46.398859 23407 raft_consensus.cc:1261] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Refusing update from remote peer 4ffc59b0e3a24f2cae42442c9d86f4f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:22:46.399664 23926 consensus_queue.cc:1035] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:46.399858 23922 consensus_queue.cc:1035] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:22:46.407538 23932 mvcc.cc:205] Tried to move back new op lower bound from 7316875535961436160 to 7316875535713988608. Current Snapshot: MvccSnapshot[applied={T|T < 7316875535961436160}]
I20260810 07:22:46.417636 23181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:55012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:46.417798 23181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:55012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:22:46.420322 23181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Sending DeleteTablet for 3 replicas of tablet 05e9dc6291894611a6449c6d447aca8d
I20260810 07:22:46.421149 23585 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e9dc6291894611a6449c6d447aca8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:46 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:37458
I20260810 07:22:46.421252 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 23252
I20260810 07:22:46.421339 23373 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e9dc6291894611a6449c6d447aca8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:46 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:38988
I20260810 07:22:46.421391 23782 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e9dc6291894611a6449c6d447aca8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:22:46 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:42762
I20260810 07:22:46.424026 23945 tablet_replica.cc:330] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: stopping tablet replica
I20260810 07:22:46.424444 23945 raft_consensus.cc:2229] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:22:46.424883 23945 raft_consensus.cc:2258] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:46.427343 23945 ts_tablet_manager.cc:1905] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:46.430493 23945 ts_tablet_manager.cc:1918] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:22:46.430725 23945 log.cc:1196] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/05e9dc6291894611a6449c6d447aca8d
I20260810 07:22:46.431172 23945 ts_tablet_manager.cc:1939] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Deleting consensus metadata
I20260810 07:22:46.432778 23154 catalog_manager.cc:4841] TS 580b02fcc6e74a12814b4f8e0ee51d3b (127.1.125.195:43925): tablet 05e9dc6291894611a6449c6d447aca8d (table test-table [id=8eaec630c8844e15bfae66cf670d9387]) successfully deleted
I20260810 07:22:46.436803 23944 tablet_replica.cc:330] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: stopping tablet replica
I20260810 07:22:46.437242 23944 raft_consensus.cc:2229] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:22:46.437783 23944 raft_consensus.cc:2258] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:22:46.439692 23944 ts_tablet_manager.cc:1905] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:22:46.441788 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 23475
W20260810 07:22:46.441780 23154 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:32849 (error 108)
I20260810 07:22:46.448235 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.447702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:32849 (local address 127.0.0.1:39004)
0810 07:22:46.447988 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:46.448101 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:32849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
W20260810 07:22:46.449316 23154 catalog_manager.cc:4584] TS 989c791077be44fb912e71852395a9c8 (127.1.125.193:32849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05e9dc6291894611a6449c6d447aca8d: Network error: Client connection negotiation failed: client connection to 127.1.125.193:32849: connect: Connection refused (error 111)
I20260810 07:22:46.459728 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 23689
I20260810 07:22:46.461688 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:46.460001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:42571 (local address 127.0.0.1:37472)
0810 07:22:46.461479 (+ 1478us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:46.461561 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.194:42571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1348}
I20260810 07:22:46.473342 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 23129
I20260810 07:22:46.488982 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38631
--webserver_interface=127.1.125.254
--webserver_port=46759
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:38631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:22:46.605850 23947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:46.606117 23947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:46.606158 23947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:46.610241 23947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:22:46.610303 23947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:46.610322 23947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:46.610339 23947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:22:46.610355 23947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:22:46.615662 23947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:38631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=46759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:46.616850 23947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:46.617918 23947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:46.626869 23954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:46.626864 23957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:46.626852 23955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:46.627143 23947 server_base.cc:1034] running on GCE node
I20260810 07:22:46.627826 23947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:46.629071 23947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:46.630266 23947 hybrid_clock.cc:648] HybridClock initialized: now 1786346566630238 us; error 43 us; skew 500 ppm
Aug 10 07:22:46 dist-test-slave-44d2 krb5kdc[22307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:46.633095 23947 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:22:46.633980 23947 webserver.cc:460] Webserver started at http://127.1.125.254:46759/ using document root <none> and password file <none>
I20260810 07:22:46.634517 23947 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:46.634577 23947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:46.638447 23947 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:22:46.640538 23970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:46.641829 23947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:22:46.641966 23947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "7b98ed3a20a54625baa78c3364cb116c"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "b4bf7a3723f5e54ece93a42848032538"
server_key_iv: "46488786a67e21e6942942e185d23ca4"
server_key_version: "encryptionkey@0"
I20260810 07:22:46.642496 23947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:46.656462 23947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:46.659670 23947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:46.659839 23947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:46.667843 23947 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:38631
I20260810 07:22:46.668130 24054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:38631 every 8 connection(s)
I20260810 07:22:46.669144 23947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:22:46.673731 24055 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:22:46.675467 24055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Bootstrap starting.
I20260810 07:22:46.676774 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 23947
I20260810 07:22:46.677258 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:32849
--local_ip_for_outbound_sockets=127.1.125.193
--tserver_master_addrs=127.1.125.254:38631
--webserver_port=41447
--webserver_interface=127.1.125.193
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:46.699903 24055 log.cc:824] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:46.715327 24055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:46.715824 24055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c: Bootstrap complete.
I20260810 07:22:46.721587 24055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } }
I20260810 07:22:46.722110 24055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b98ed3a20a54625baa78c3364cb116c, State: Initialized, Role: FOLLOWER
I20260810 07:22:46.722682 24055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } }
I20260810 07:22:46.722844 24055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:22:46.722926 24055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:22:46.723037 24055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:22:46.725708 24055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } }
I20260810 07:22:46.726070 24055 leader_election.cc:302] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b98ed3a20a54625baa78c3364cb116c; no voters:
I20260810 07:22:46.726451 24055 leader_election.cc:288] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 07:22:46.726636 24062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 3 FOLLOWER]: Leader election won for term 3
I20260810 07:22:46.726926 24062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [term 3 LEADER]: Becoming Leader. State: Replica: 7b98ed3a20a54625baa78c3364cb116c, State: Running, Role: LEADER
I20260810 07:22:46.727247 24062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } }
I20260810 07:22:46.727828 24055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:22:46.728796 24063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7b98ed3a20a54625baa78c3364cb116c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } } }
I20260810 07:22:46.728951 24063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:46.729295 24064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b98ed3a20a54625baa78c3364cb116c. Latest consensus state: current_term: 3 leader_uuid: "7b98ed3a20a54625baa78c3364cb116c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b98ed3a20a54625baa78c3364cb116c" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38631 } } }
I20260810 07:22:46.729390 24064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b98ed3a20a54625baa78c3364cb116c [sys.catalog]: This master's current role is: LEADER
I20260810 07:22:46.729501 24073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:22:46.734872 24073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:22:46.735800 24073 catalog_manager.cc:1259] Loaded cluster ID: 668254fc929340d3b1d307aab67312fb
I20260810 07:22:46.735867 24073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 07:22:46.737028 24073 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f65e5be79ad google::LogMessage::Fail() at ??:0
@ 0x7f65e5bea88a google::LogMessage::SendToLog() at ??:0
@ 0x7f65e5be74de google::LogMessage::Flush() at ??:0
@ 0x7f65e5be8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f65f0526fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f65f0527d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f65f0524579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f65f0568640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f65e7e393d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f65e7e39c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f65e7e3b3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f65e7e2a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f65e946e6db start_thread at ??:0
@ 0x7f65e46e561f clone at ??:0
W20260810 07:22:46.832266 24058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:46.832649 24058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:46.832777 24058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:46.836968 24058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:46.837040 24058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:46.837126 24058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:22:46.842437 24058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:32849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=41447
--tserver_master_addrs=127.1.125.254:38631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:46.843551 24058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:46.844673 24058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:46.854620 24114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:46.854784 24113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:46.855032 24116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:46.855928 24058 server_base.cc:1034] running on GCE node
I20260810 07:22:46.856346 24058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:46.857075 24058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:46.858278 24058 hybrid_clock.cc:648] HybridClock initialized: now 1786346566858243 us; error 54 us; skew 500 ppm
Aug 10 07:22:46 dist-test-slave-44d2 krb5kdc[22307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:46.861344 24058 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:22:46.862337 24058 webserver.cc:460] Webserver started at http://127.1.125.193:41447/ using document root <none> and password file <none>
I20260810 07:22:46.862933 24058 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:46.863034 24058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:46.867437 24058 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:22:46.869819 24125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:46.871273 24058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 07:22:46.871433 24058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "989c791077be44fb912e71852395a9c8"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "b83cef3e6a6ca8b1f01c7f373f821a23"
server_key_iv: "5230b56facf8a0e854844ee86c999553"
server_key_version: "encryptionkey@0"
I20260810 07:22:46.872004 24058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:46.894824 24058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:46.898536 24058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:46.898792 24058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:46.899422 24058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:46.900672 24137 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:22:46.907018 24058 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:22:46.907106 24058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260810 07:22:46.907212 24058 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:22:46.909327 24058 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:22:46.909404 24058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260810 07:22:46.909790 24137 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8: Bootstrap starting.
I20260810 07:22:46.920785 24058 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:32849
I20260810 07:22:46.920969 24302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:32849 every 8 connection(s)
I20260810 07:22:46.921805 24058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:22:46.926854 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 24058
I20260810 07:22:46.927299 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:42571
--local_ip_for_outbound_sockets=127.1.125.194
--tserver_master_addrs=127.1.125.254:38631
--webserver_port=40391
--webserver_interface=127.1.125.194
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:46.934134 24137 log.cc:824] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:46.944250 24137 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:46.944962 24137 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8: Bootstrap complete.
I20260810 07:22:46.946712 24137 ts_tablet_manager.cc:1397] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8: Time spent bootstrapping tablet: real 0.037s user 0.026s sys 0.008s
I20260810 07:22:46.950537 24137 raft_consensus.cc:348] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:46.951130 24137 raft_consensus.cc:729] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989c791077be44fb912e71852395a9c8, State: Initialized, Role: FOLLOWER
I20260810 07:22:46.951812 24137 consensus_queue.cc:260] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:46.952983 24137 ts_tablet_manager.cc:1428] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8: Time spent starting tablet: real 0.006s user 0.003s sys 0.005s
W20260810 07:22:47.022545 24305 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:22:46.923939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:34281)
0810 07:22:46.924438 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:46.924519 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:46.925361 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:47.021426 (+ 96065us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":150,"threads_started":1}
W20260810 07:22:47.023162 24303 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:38631 (0 consecutive failures): Network error: Failed to ping master at 127.1.125.254:38631: Client connection negotiation failed: client connection to 127.1.125.254:38631: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:22:47.024053 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:47.023710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:48909)
0810 07:22:47.023890 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:47.023977 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 07:22:47.024945 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:47.024681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:33539)
0810 07:22:47.024810 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:47.024878 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 07:22:47.025171 24303 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:22:47.051425 24307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:47.051666 24307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:47.051738 24307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:47.055869 24307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:47.055928 24307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:47.056031 24307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:22:47.061241 24307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:42571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=40391
--tserver_master_addrs=127.1.125.254:38631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:47.062312 24307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:47.063350 24307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:47.073375 24322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:47.073379 24327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:47.073386 24323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:47.073832 24307 server_base.cc:1034] running on GCE node
I20260810 07:22:47.074239 24307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:47.075002 24307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:47.076220 24307 hybrid_clock.cc:648] HybridClock initialized: now 1786346567076191 us; error 48 us; skew 500 ppm
Aug 10 07:22:47 dist-test-slave-44d2 krb5kdc[22307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346567, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:47.079056 24307 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:22:47.080005 24307 webserver.cc:460] Webserver started at http://127.1.125.194:40391/ using document root <none> and password file <none>
I20260810 07:22:47.080657 24307 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:47.080755 24307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:47.084862 24307 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:22:47.087311 24338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:47.088682 24307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:22:47.088819 24307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "ec8b94c361a342bea5ee5ae6ece95433"
server_key_iv: "ee93e6f5d848468269281871d5915e81"
server_key_version: "encryptionkey@0"
I20260810 07:22:47.089392 24307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:47.114651 24307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:47.117719 24307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:47.117938 24307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:47.118592 24307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:47.119724 24346 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:22:47.123983 24307 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:22:47.124065 24307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260810 07:22:47.124140 24307 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:22:47.125852 24307 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:22:47.125926 24307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260810 07:22:47.126228 24346 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: Bootstrap starting.
I20260810 07:22:47.131903 24346 log.cc:824] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:47.137320 24307 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:42571
I20260810 07:22:47.137391 24506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:42571 every 8 connection(s)
I20260810 07:22:47.138423 24307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:22:47.141287 24346 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:47.141721 24346 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: Bootstrap complete.
I20260810 07:22:47.142092 24346 ts_tablet_manager.cc:1397] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
I20260810 07:22:47.142364 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:47.140887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:38047)
0810 07:22:47.141461 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:47.141590 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":162,"threads_started":1}
W20260810 07:22:47.142802 24507 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:38631 (0 consecutive failures): Network error: Failed to ping master at 127.1.125.254:38631: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
I20260810 07:22:47.143399 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:47.143164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:57289)
0810 07:22:47.143281 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:47.143341 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 07:22:47.144111 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:47.143908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:59005)
0810 07:22:47.144013 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:47.144055 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 07:22:47.144352 24507 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:22:47.145349 24346 raft_consensus.cc:348] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:47.145771 24346 raft_consensus.cc:729] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ffc59b0e3a24f2cae42442c9d86f4f6, State: Initialized, Role: FOLLOWER
I20260810 07:22:47.146268 24346 consensus_queue.cc:260] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:47.146495 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 24307
I20260810 07:22:47.147039 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:43925
--local_ip_for_outbound_sockets=127.1.125.195
--tserver_master_addrs=127.1.125.254:38631
--webserver_port=37381
--webserver_interface=127.1.125.195
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:22:47.147284 24346 ts_tablet_manager.cc:1428] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:22:47.147454 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:47.147144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:36497)
0810 07:22:47.147325 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:47.147382 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260810 07:22:47.148065 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:47.147857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:57599)
0810 07:22:47.147970 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:47.148020 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 07:22:47.264861 24513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:22:47.265116 24513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:22:47.265190 24513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:22:47.269658 24513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:22:47.269728 24513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:22:47.269820 24513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:22:47.275218 24513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:43925
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=37381
--tserver_master_addrs=127.1.125.254:38631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:22:47.276422 24513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:22:47.277504 24513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:22:47.287992 24524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:47.288002 24521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:22:47.288002 24522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:22:47.288079 24513 server_base.cc:1034] running on GCE node
I20260810 07:22:47.288702 24513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:22:47.289356 24513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:22:47.290678 24513 hybrid_clock.cc:648] HybridClock initialized: now 1786346567290656 us; error 36 us; skew 500 ppm
Aug 10 07:22:47 dist-test-slave-44d2 krb5kdc[22307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346567, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:22:47.293567 24513 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:22:47.294495 24513 webserver.cc:460] Webserver started at http://127.1.125.195:37381/ using document root <none> and password file <none>
I20260810 07:22:47.295027 24513 fs_manager.cc:359] Metadata directory not provided
I20260810 07:22:47.295086 24513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:22:47.298906 24513 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:22:47.300971 24537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:47.302237 24513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:22:47.302386 24513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "580b02fcc6e74a12814b4f8e0ee51d3b"
format_stamp: "Formatted at 2026-08-10 07:22:44 on dist-test-slave-44d2"
server_key: "dc735303668e334a1c46e8b80a7710e9"
server_key_iv: "ebcf4d1164674128a4f5d9a5ed975911"
server_key_version: "encryptionkey@0"
I20260810 07:22:47.302961 24513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:22:47.327623 24513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:22:47.331239 24513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:22:47.331475 24513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:22:47.332163 24513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:22:47.333106 24513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:22:47.333180 24513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:47.333256 24513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:22:47.333309 24513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:22:47.342664 24513 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:43925
I20260810 07:22:47.342698 24706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:43925 every 8 connection(s)
I20260810 07:22:47.343693 24513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:22:47.344861 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 24513
I20260810 07:22:47.348138 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:47.346094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:39019)
0810 07:22:47.346692 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:47.346851 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":172,"threads_started":1}
W20260810 07:22:47.348675 24707 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:38631 (0 consecutive failures): Network error: Failed to ping master at 127.1.125.254:38631: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
I20260810 07:22:47.349231 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:47.349013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:49015)
0810 07:22:47.349128 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:47.349175 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:22:47.349855 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:47.349637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:35699)
0810 07:22:47.349749 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:47.349795 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 07:22:47.350040 24707 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:22:48.026345 24717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:48.025832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:56021)
0810 07:22:48.026161 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:48.026219 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":108,"threads_started":1}
I20260810 07:22:48.149209 24718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:48.148812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:60905)
0810 07:22:48.149058 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:48.149111 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":71,"threads_started":1}
I20260810 07:22:48.351102 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:48.350682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:44757)
0810 07:22:48.350944 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:48.350999 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":90,"threads_started":1}
I20260810 07:22:48.467144 24720 raft_consensus.cc:482] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:22:48.467361 24720 raft_consensus.cc:504] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:48.468545 24720 leader_election.cc:288] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4ffc59b0e3a24f2cae42442c9d86f4f6 (127.1.125.194:42571), 580b02fcc6e74a12814b4f8e0ee51d3b (127.1.125.195:43925)
Aug 10 07:22:48 dist-test-slave-44d2 krb5kdc[22307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.194@KRBTEST.COM
Aug 10 07:22:48 dist-test-slave-44d2 krb5kdc[22307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.195@KRBTEST.COM
I20260810 07:22:48.483186 24717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:48.468726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:42571 (local address 127.1.125.193:43883)
0810 07:22:48.468839 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:48.468858 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:48.469058 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:48.470355 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:48.470361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:48.470913 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:48.471518 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:48.471533 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:48.473069 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:48.473072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:48.473927 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:48.473938 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:48.474913 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:48.474917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:48.475129 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:48.475790 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:48.475816 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:48.477563 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:48.480050 (+ 2487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:48.480055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:48.480068 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:48.480505 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:48.480944 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:48.480947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:48.480949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:48.481182 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:48.481958 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:48.481965 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:48.482355 (+ 390us) client_negotiation.cc:769] Sending connection context
0810 07:22:48.482592 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 07:22:48.483025 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":281,"spinlock_wait_cycles":18048}
I20260810 07:22:48.483232 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:48.468902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:43883 (local address 127.1.125.194:42571)
0810 07:22:48.469338 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:48.469354 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:48.469410 (+ 56us) server_negotiation.cc:407] Connection header received
0810 07:22:48.469644 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:48.469649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:48.469984 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:48.470228 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:48.471728 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:48.472928 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:48.474065 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:48.474749 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:48.474950 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:48.477792 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:48.477797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:48.477811 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:48.477839 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:48.479928 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:48.480631 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:48.480636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:48.480641 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:48.480817 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:48.481293 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:48.481296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:48.481298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:48.481847 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:48.481970 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:48.482733 (+ 763us) server_negotiation.cc:299] Negotiation successful
0810 07:22:48.483025 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":103,"threads_started":1}
I20260810 07:22:48.483186 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:48.468834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:43925 (local address 127.1.125.193:45119)
0810 07:22:48.469189 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:48.469205 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:48.469346 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:48.471012 (+ 1666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:48.471015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:48.471194 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:48.471516 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:48.471532 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:48.472953 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:48.472956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:48.473930 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:48.473941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:48.475137 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:48.475141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:48.475226 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:48.475926 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:48.475939 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:48.477917 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:48.480290 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:48.480293 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:48.480295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:48.480604 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:48.480944 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:48.480947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:48.480949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:48.481182 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:48.481958 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:48.481965 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:48.482355 (+ 390us) client_negotiation.cc:769] Sending connection context
0810 07:22:48.482594 (+ 239us) client_negotiation.cc:240] Negotiation successful
0810 07:22:48.482971 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":171,"spinlock_wait_cycles":12288,"thread_start_us":74,"threads_started":1}
I20260810 07:22:48.483218 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:48.469088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:45119 (local address 127.1.125.195:43925)
0810 07:22:48.469483 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:48.469504 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:48.469537 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:22:48.470256 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:48.470265 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:48.470639 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:48.470933 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:48.471695 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:48.472801 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:48.474289 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:48.474953 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:48.475155 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:48.478133 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:48.478143 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:48.478161 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:48.478200 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:48.480178 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:48.480709 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:48.480717 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:48.480724 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:48.480846 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:48.481293 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:48.481296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:48.481298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:48.481824 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:48.481970 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:48.482733 (+ 763us) server_negotiation.cc:299] Negotiation successful
0810 07:22:48.483041 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":149,"threads_started":1}
I20260810 07:22:48.484737 24440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e9dc6291894611a6449c6d447aca8d" candidate_uuid: "989c791077be44fb912e71852395a9c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" is_pre_election: true
I20260810 07:22:48.485128 24440 raft_consensus.cc:2454] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 989c791077be44fb912e71852395a9c8 in term 1.
I20260810 07:22:48.486236 24127 leader_election.cc:302] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ffc59b0e3a24f2cae42442c9d86f4f6, 989c791077be44fb912e71852395a9c8; no voters:
W20260810 07:22:48.486434 24130 leader_election.cc:341] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 580b02fcc6e74a12814b4f8e0ee51d3b (127.1.125.195:43925): Not found: Tablet not found: 05e9dc6291894611a6449c6d447aca8d
I20260810 07:22:48.486531 24720 raft_consensus.cc:2781] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:22:48.486652 24720 raft_consensus.cc:482] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:22:48.486727 24720 raft_consensus.cc:3037] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:22:48.488965 24720 raft_consensus.cc:504] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:48.489338 24720 leader_election.cc:288] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [CANDIDATE]: Term 2 election: Requested vote from peers 4ffc59b0e3a24f2cae42442c9d86f4f6 (127.1.125.194:42571), 580b02fcc6e74a12814b4f8e0ee51d3b (127.1.125.195:43925)
I20260810 07:22:48.489876 24440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e9dc6291894611a6449c6d447aca8d" candidate_uuid: "989c791077be44fb912e71852395a9c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6"
W20260810 07:22:48.489961 24130 leader_election.cc:341] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 580b02fcc6e74a12814b4f8e0ee51d3b (127.1.125.195:43925): Not found: Tablet not found: 05e9dc6291894611a6449c6d447aca8d
I20260810 07:22:48.490041 24440 raft_consensus.cc:3037] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:22:48.491976 24440 raft_consensus.cc:2454] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 989c791077be44fb912e71852395a9c8 in term 2.
I20260810 07:22:48.492324 24127 leader_election.cc:302] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ffc59b0e3a24f2cae42442c9d86f4f6, 989c791077be44fb912e71852395a9c8; no voters: 580b02fcc6e74a12814b4f8e0ee51d3b
I20260810 07:22:48.492523 24720 raft_consensus.cc:2781] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:22:48.492735 24720 raft_consensus.cc:686] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [term 2 LEADER]: Becoming Leader. State: Replica: 989c791077be44fb912e71852395a9c8, State: Running, Role: LEADER
I20260810 07:22:48.493096 24720 consensus_queue.cc:237] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:48.493499 24717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:48.493218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:54081)
0810 07:22:48.493371 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:48.493422 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260810 07:22:48.894203 24130 consensus_peers.cc:487] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 -> Peer 580b02fcc6e74a12814b4f8e0ee51d3b (127.1.125.195:43925): Couldn't send request to peer 580b02fcc6e74a12814b4f8e0ee51d3b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 05e9dc6291894611a6449c6d447aca8d. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:22:49.016049 24440 raft_consensus.cc:1261] T 05e9dc6291894611a6449c6d447aca8d P 4ffc59b0e3a24f2cae42442c9d86f4f6 [term 2 FOLLOWER]: Refusing update from remote peer 989c791077be44fb912e71852395a9c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260810 07:22:49.016880 24720 consensus_queue.cc:1035] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260810 07:22:49.019119 24730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:49.018704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:54247)
0810 07:22:49.018924 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:49.019015 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":61,"threads_started":1}
I20260810 07:22:49.021780 24736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:49.021357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:51059)
0810 07:22:49.021612 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:49.021673 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":71,"threads_started":1}
I20260810 07:22:49.021869 24730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:49.021623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:36605)
0810 07:22:49.021726 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:49.021784 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 07:22:49.352303 24737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:49.351845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:46169)
0810 07:22:49.352133 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:49.352191 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":100,"threads_started":1}
I20260810 07:22:49.454934 24739 ts_tablet_manager.cc:927] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Initiating tablet copy from peer 989c791077be44fb912e71852395a9c8 (127.1.125.193:32849)
I20260810 07:22:49.455704 24739 tablet_copy_client.cc:316] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: tablet copy: Beginning tablet copy session from remote peer at address 127.1.125.193:32849
Aug 10 07:22:49 dist-test-slave-44d2 krb5kdc[22307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346567, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.193@KRBTEST.COM
I20260810 07:22:49.467098 24737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:49.456314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:32849 (local address 127.1.125.195:51697)
0810 07:22:49.456458 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:49.456515 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 07:22:49.456667 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:22:49.457444 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:22:49.457451 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:22:49.457910 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:22:49.458042 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:49.458049 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:49.459031 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:49.459034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:49.459699 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:22:49.459708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:22:49.460342 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:22:49.460345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:22:49.460465 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:49.461162 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:22:49.461175 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:22:49.462748 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:22:49.465039 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:49.465049 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:49.465056 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:49.465332 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:49.465702 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:22:49.465704 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:22:49.465705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:22:49.465837 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:22:49.466434 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:22:49.466441 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:22:49.466564 (+ 123us) client_negotiation.cc:769] Sending connection context
0810 07:22:49.466693 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 07:22:49.466958 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 07:22:49.467098 24741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:49.456502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:51697 (local address 127.1.125.193:32849)
0810 07:22:49.456868 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0810 07:22:49.456878 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:22:49.456899 (+ 21us) server_negotiation.cc:407] Connection header received
0810 07:22:49.456946 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:22:49.456951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:22:49.457206 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:22:49.457370 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:22:49.458166 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:49.458908 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:49.459812 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:22:49.460173 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:22:49.460259 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:22:49.462910 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:22:49.462921 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:22:49.462934 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:22:49.462963 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:22:49.464801 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:49.465452 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:49.465458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:49.465459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:49.465609 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:22:49.465966 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:22:49.465969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:22:49.465970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:22:49.466317 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:22:49.466435 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 07:22:49.466791 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 07:22:49.466938 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":95,"threads_started":1}
I20260810 07:22:49.468578 24256 tablet_copy_service.cc:140] P 989c791077be44fb912e71852395a9c8: Received BeginTabletCopySession request for tablet 05e9dc6291894611a6449c6d447aca8d from peer 580b02fcc6e74a12814b4f8e0ee51d3b ({username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:51697)
I20260810 07:22:49.468815 24256 tablet_copy_service.cc:161] P 989c791077be44fb912e71852395a9c8: Beginning new tablet copy session on tablet 05e9dc6291894611a6449c6d447aca8d from peer 580b02fcc6e74a12814b4f8e0ee51d3b at {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:51697: session id = 580b02fcc6e74a12814b4f8e0ee51d3b-05e9dc6291894611a6449c6d447aca8d
I20260810 07:22:49.470139 24256 tablet_copy_source_session.cc:205] T 05e9dc6291894611a6449c6d447aca8d P 989c791077be44fb912e71852395a9c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 07:22:49.474823 24739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e9dc6291894611a6449c6d447aca8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:22:49.481662 24739 tablet_copy_client.cc:791] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: tablet copy: Starting download of 0 data blocks...
I20260810 07:22:49.481932 24739 tablet_copy_client.cc:655] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: tablet copy: Starting download of 1 WAL segments...
I20260810 07:22:49.483994 24739 tablet_copy_client.cc:531] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 07:22:49.485992 24739 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Bootstrap starting.
I20260810 07:22:49.492460 24739 log.cc:824] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:22:49.499703 24739 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:22:49.500030 24739 tablet_bootstrap.cc:492] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Bootstrap complete.
I20260810 07:22:49.500386 24739 ts_tablet_manager.cc:1397] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260810 07:22:49.502981 24739 raft_consensus.cc:348] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:49.503265 24739 raft_consensus.cc:729] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580b02fcc6e74a12814b4f8e0ee51d3b, State: Initialized, Role: FOLLOWER
I20260810 07:22:49.503775 24739 consensus_queue.cc:260] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffc59b0e3a24f2cae42442c9d86f4f6" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 42571 } } peers { permanent_uuid: "580b02fcc6e74a12814b4f8e0ee51d3b" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 43925 } } peers { permanent_uuid: "989c791077be44fb912e71852395a9c8" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 32849 } }
I20260810 07:22:49.504814 24739 ts_tablet_manager.cc:1428] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b: Time spent starting tablet: real 0.004s user 0.007s sys 0.001s
I20260810 07:22:49.505092 24737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:49.504807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:49183)
0810 07:22:49.504947 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:49.505008 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 07:22:49.505631 24256 tablet_copy_service.cc:342] P 989c791077be44fb912e71852395a9c8: Request end of tablet copy session 580b02fcc6e74a12814b4f8e0ee51d3b-05e9dc6291894611a6449c6d447aca8d received from {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:51697
I20260810 07:22:49.505765 24256 tablet_copy_service.cc:433] P 989c791077be44fb912e71852395a9c8: ending tablet copy session 580b02fcc6e74a12814b4f8e0ee51d3b-05e9dc6291894611a6449c6d447aca8d on tablet 05e9dc6291894611a6449c6d447aca8d with peer 580b02fcc6e74a12814b4f8e0ee51d3b
I20260810 07:22:49.505892 24737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:49.505606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:33839)
0810 07:22:49.505720 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:49.505799 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 07:22:50.022863 24747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:50.022389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:60733)
0810 07:22:50.022702 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:50.022768 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":60,"threads_started":1}
I20260810 07:22:50.022927 24748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:50.022509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:58961)
0810 07:22:50.022747 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:50.022814 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":66,"threads_started":1}
I20260810 07:22:50.022876 24640 raft_consensus.cc:1203] T 05e9dc6291894611a6449c6d447aca8d P 580b02fcc6e74a12814b4f8e0ee51d3b [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260810 07:22:50.024124 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:50.023781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:48803)
0810 07:22:50.023966 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:50.024025 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":50,"threads_started":1}
I20260810 07:22:51.024459 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:51.023885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:50881)
0810 07:22:51.024241 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:51.024313 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":85,"threads_started":1}
I20260810 07:22:51.024453 24752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:51.023885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:48163)
0810 07:22:51.024304 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:51.024362 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":120,"threads_started":1}
I20260810 07:22:51.025308 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:51.024886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:33759)
0810 07:22:51.025178 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:51.025230 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":71,"threads_started":1}
I20260810 07:22:52.025934 24755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:52.025372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:41293)
0810 07:22:52.025720 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:52.025799 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":95,"threads_started":1}
I20260810 07:22:52.026153 24756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:52.025684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:39375)
0810 07:22:52.025973 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:52.026037 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":82,"threads_started":1}
I20260810 07:22:52.026499 24757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:52.026066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:37719)
0810 07:22:52.026342 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:52.026403 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":71,"threads_started":1}
I20260810 07:22:52.027002 24756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:52.026708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:38365)
0810 07:22:52.026883 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:52.026934 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260810 07:22:53.027424 24759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:53.026889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:54219)
0810 07:22:53.027184 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:53.027301 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":93,"threads_started":1}
I20260810 07:22:53.027920 24760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:53.027474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:41327)
0810 07:22:53.027774 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:53.027839 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":78,"threads_started":1}
I20260810 07:22:53.028170 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:53.027700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:57603)
0810 07:22:53.027956 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:53.028043 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260810 07:22:54.029261 24764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:54.028819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:36905)
0810 07:22:54.029126 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:54.029174 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":66,"threads_started":1}
I20260810 07:22:54.029244 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:54.028526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:59989)
0810 07:22:54.028958 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:54.029119 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":111,"threads_started":1}
I20260810 07:22:54.029476 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:54.028931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:52119)
0810 07:22:54.029309 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:54.029371 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":95,"threads_started":1}
I20260810 07:22:55.030556 24770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:55.030076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:36091)
0810 07:22:55.030344 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:55.030410 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":99,"threads_started":1}
I20260810 07:22:55.030614 24775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:55.030205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:56257)
0810 07:22:55.030470 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:55.030523 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":83,"threads_started":1}
I20260810 07:22:55.030640 24772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:55.030126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:36995)
0810 07:22:55.030453 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:55.030530 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":76,"threads_started":1}
I20260810 07:22:56.031925 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:56.031487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:60333)
0810 07:22:56.031747 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:56.031807 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":95,"threads_started":1}
I20260810 07:22:56.031985 24781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:56.031456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:55771)
0810 07:22:56.031842 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:56.031897 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":100,"threads_started":1}
I20260810 07:22:56.032022 24783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:56.031553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:43021)
0810 07:22:56.031837 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:56.031892 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":91,"threads_started":1}
I20260810 07:22:57.033445 24789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:57.033062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:50493)
0810 07:22:57.033294 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:57.033350 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":65,"threads_started":1}
I20260810 07:22:57.033448 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:57.032881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:42297)
0810 07:22:57.033289 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:57.033340 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":99,"threads_started":1}
I20260810 07:22:57.033448 24787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:57.032869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:44781)
0810 07:22:57.033277 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:57.033340 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":106,"threads_started":1}
I20260810 07:22:58.034833 24792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.034287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:52733)
0810 07:22:58.034690 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:58.034743 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":74,"threads_started":1}
I20260810 07:22:58.034819 24794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.034346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:43267)
0810 07:22:58.034643 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:58.034702 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":86,"threads_started":1}
I20260810 07:22:58.034817 24793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:58.034346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:59005)
0810 07:22:58.034639 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:58.034702 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":89,"threads_started":1}
I20260810 07:22:59.036144 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:59.035704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:57129)
0810 07:22:59.035984 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:59.036040 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":105,"threads_started":1}
I20260810 07:22:59.036207 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:59.035700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:44983)
0810 07:22:59.036039 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:59.036100 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":83,"threads_started":1}
I20260810 07:22:59.036332 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:22:59.035862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:46265)
0810 07:22:59.036177 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 07:22:59.036240 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":68,"threads_started":1}
I20260810 07:23:00.037595 24802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:00.037072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:40423)
0810 07:23:00.037397 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:00.037488 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":95,"threads_started":1}
I20260810 07:23:00.037652 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:00.037123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:44765)
0810 07:23:00.037484 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:00.037540 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":108,"threads_started":1}
I20260810 07:23:00.037647 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:00.037073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:40983)
0810 07:23:00.037466 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:00.037536 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":88,"threads_started":1}
I20260810 07:23:01.038893 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:01.038356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:55129)
0810 07:23:01.038661 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:01.038726 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":99,"threads_started":1}
I20260810 07:23:01.038937 24811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:01.038507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:52271)
0810 07:23:01.038784 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:01.038839 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":71,"threads_started":1}
I20260810 07:23:01.038925 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:01.038420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:60837)
0810 07:23:01.038748 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:01.038829 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":120,"threads_started":1}
I20260810 07:23:02.040385 24813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:02.039904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.193:40003)
0810 07:23:02.040224 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:02.040284 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":86,"threads_started":1}
I20260810 07:23:02.040385 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:02.039834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.194:42301)
0810 07:23:02.040225 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:02.040284 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":182,"threads_started":1}
I20260810 07:23:02.040372 24812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:02.039853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38631 (local address 127.1.125.195:54373)
0810 07:23:02.040192 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:02.040265 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:38631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":97,"threads_started":1}
I20260810 07:23:02.350517 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 24058
I20260810 07:23:02.367173 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 24307
I20260810 07:23:02.376222 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 24513
2026-08-10T07:23:02Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19684 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:23:02 dist-test-slave-44d2 krb5kdc[24820](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:23:02 dist-test-slave-44d2 krb5kdc[24820](info): set up 2 sockets
Aug 10 07:23:02 dist-test-slave-44d2 krb5kdc[24820](info): commencing operation
krb5kdc: starting...
W20260810 07:23:03.677831 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:23:03 dist-test-slave-44d2 krb5kdc[24820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:23:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:23:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:23:03.809459 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:34003
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:40763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:34003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:03.923233 24842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:03.923547 24842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:03.923627 24842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:03.927838 24842 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 07:23:03.927918 24842 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 07:23:03.927949 24842 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 07:23:03.927984 24842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:03.928028 24842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:03.928069 24842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:03.928105 24842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:03.928141 24842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:03.933343 24842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:40763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:34003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:34003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:23:03.934412 24842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:03.935454 24842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:03.944363 24852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:03.944346 24842 server_base.cc:1034] running on GCE node
W20260810 07:23:03.944322 24855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:03.944346 24850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:03.945312 24842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:03.946393 24842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:03.947558 24842 hybrid_clock.cc:648] HybridClock initialized: now 1786346583947542 us; error 35 us; skew 500 ppm
Aug 10 07:23:03 dist-test-slave-44d2 krb5kdc[24820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346583, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:03.950327 24842 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:23:03.951246 24842 webserver.cc:460] Webserver started at http://127.1.125.254:42147/ using document root <none> and password file <none>
I20260810 07:23:03.951807 24842 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:03.951901 24842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:03.952163 24842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:03.954268 24842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "4978926106d341a6a9962256304bc518"
format_stamp: "Formatted at 2026-08-10 07:23:03 on dist-test-slave-44d2"
server_key: "5a594dc1fe586cbbdf2fff07c816a52b"
server_key_iv: "42513ba2ffab905ed844d0351985cf73"
server_key_version: "encryptionkey@0"
I20260810 07:23:03.954840 24842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "4978926106d341a6a9962256304bc518"
format_stamp: "Formatted at 2026-08-10 07:23:03 on dist-test-slave-44d2"
server_key: "5a594dc1fe586cbbdf2fff07c816a52b"
server_key_iv: "42513ba2ffab905ed844d0351985cf73"
server_key_version: "encryptionkey@0"
I20260810 07:23:03.958607 24842 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:23:03.961051 24864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:03.962281 24842 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:23:03.962430 24842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "4978926106d341a6a9962256304bc518"
format_stamp: "Formatted at 2026-08-10 07:23:03 on dist-test-slave-44d2"
server_key: "5a594dc1fe586cbbdf2fff07c816a52b"
server_key_iv: "42513ba2ffab905ed844d0351985cf73"
server_key_version: "encryptionkey@0"
I20260810 07:23:03.962541 24842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:03.987588 24842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:03.990581 24842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:03.990801 24842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:03.998400 24842 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:34003
I20260810 07:23:03.998715 24940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:34003 every 8 connection(s)
I20260810 07:23:03.999401 24842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:04.002345 24941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:04.006363 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 24842
I20260810 07:23:04.006491 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:23:04.006845 1527 external_mini_cluster.cc:1428] Setting key 707367ebd4724691f505d52de23c8f01
I20260810 07:23:04.008662 24941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Bootstrap starting.
I20260810 07:23:04.010975 24941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:04.011757 24941 log.cc:824] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:04.013731 24941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: No bootstrap required, opened a new log
Aug 10 07:23:04 dist-test-slave-44d2 krb5kdc[24820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:23:04.018072 24941 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } }
I20260810 07:23:04.018292 24941 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:04.018328 24941 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4978926106d341a6a9962256304bc518, State: Initialized, Role: FOLLOWER
I20260810 07:23:04.018838 24941 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } }
I20260810 07:23:04.018980 24941 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:04.019062 24941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:04.019167 24941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:04.020020 24941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } }
I20260810 07:23:04.020419 24941 leader_election.cc:302] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4978926106d341a6a9962256304bc518; no voters:
I20260810 07:23:04.020838 24941 leader_election.cc:288] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:23:04.020910 24947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:04.021111 24947 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 1 LEADER]: Becoming Leader. State: Replica: 4978926106d341a6a9962256304bc518, State: Running, Role: LEADER
I20260810 07:23:04.021483 24947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } }
I20260810 07:23:04.021631 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.008303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47996 (local address 127.1.125.254:34003)
0810 07:23:04.008761 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:04.008776 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:04.008839 (+ 63us) server_negotiation.cc:407] Connection header received
0810 07:23:04.009515 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:04.009531 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:04.009850 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:04.010184 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:04.011006 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.012077 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.013047 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.013516 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.013698 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.016468 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:04.016474 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:04.016535 (+ 61us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:04.016571 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:04.018365 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:04.018816 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:04.018829 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:04.018834 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:04.018996 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:04.019322 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:04.019325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:04.019327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:04.019721 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:04.019943 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:04.020203 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 07:23:04.020542 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":159,"threads_started":1}
I20260810 07:23:04.022161 24941 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:04.023713 24947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4978926106d341a6a9962256304bc518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } } }
I20260810 07:23:04.024044 24947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:04.025121 24948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4978926106d341a6a9962256304bc518. Latest consensus state: current_term: 1 leader_uuid: "4978926106d341a6a9962256304bc518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } } }
I20260810 07:23:04.025239 24948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: This master's current role is: LEADER
W20260810 07:23:04.027292 24969 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:23:04.027423 24969 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:23:04.027503 24967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:04.028221 24967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:04.033365 24967 catalog_manager.cc:1347] Generated new cluster ID: 179fd7238e484fe7b99425dc87612281
I20260810 07:23:04.033439 24967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:04.063511 24967 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:23:04.064630 24967 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:04.069772 24967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Generated new TSK 0
I20260810 07:23:04.070446 24967 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:23:04.108438 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:34003
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:40763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:04.229228 24977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:04.229490 24977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:04.229538 24977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:04.233626 24977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:04.233698 24977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:04.233762 24977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:23:04.239065 24977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:40763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:23:04.240150 24977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:04.241199 24977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:04.251659 24984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:04.251696 24985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:04.251677 24992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:04.251859 24977 server_base.cc:1034] running on GCE node
I20260810 07:23:04.252542 24977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:04.253176 24977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:04.254393 24977 hybrid_clock.cc:648] HybridClock initialized: now 1786346584254362 us; error 35 us; skew 500 ppm
Aug 10 07:23:04 dist-test-slave-44d2 krb5kdc[24820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346584, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:04.257648 24977 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:23:04.258709 24977 webserver.cc:460] Webserver started at http://127.1.125.193:44927/ using document root <none> and password file <none>
I20260810 07:23:04.259323 24977 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:04.259390 24977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:04.259579 24977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:04.261705 24977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "8aa7968bd68a4b3eb023a62bcad8df51"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "13fff1810b5ef56d60e97c9428535813"
server_key_iv: "d0d7ac9b704b4026d73ef4bee48e6754"
server_key_version: "encryptionkey@0"
I20260810 07:23:04.262233 24977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "8aa7968bd68a4b3eb023a62bcad8df51"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "13fff1810b5ef56d60e97c9428535813"
server_key_iv: "d0d7ac9b704b4026d73ef4bee48e6754"
server_key_version: "encryptionkey@0"
I20260810 07:23:04.266227 24977 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:23:04.268883 25002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:04.270234 24977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:23:04.270345 24977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "8aa7968bd68a4b3eb023a62bcad8df51"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "13fff1810b5ef56d60e97c9428535813"
server_key_iv: "d0d7ac9b704b4026d73ef4bee48e6754"
server_key_version: "encryptionkey@0"
I20260810 07:23:04.270433 24977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:04.294070 24977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:04.297580 24977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:04.297755 24977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:04.298316 24977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:04.299160 24977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:04.299216 24977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:04.299256 24977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:04.299273 24977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:04.309530 24977 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:36935
I20260810 07:23:04.309561 25177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:36935 every 8 connection(s)
I20260810 07:23:04.310461 24977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:23:04.316473 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 24977
I20260810 07:23:04.316674 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:23:04.316969 1527 external_mini_cluster.cc:1428] Setting key 39d5dbab2174df474ac356be02797239
Aug 10 07:23:04 dist-test-slave-44d2 krb5kdc[24820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346584, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:23:04.326314 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.312430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:46083 (local address 127.1.125.254:34003)
0810 07:23:04.312631 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:04.312635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:04.313784 (+ 1149us) server_negotiation.cc:407] Connection header received
0810 07:23:04.314732 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:04.314734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:04.314784 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:04.314865 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:04.316473 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.317293 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.318756 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.318991 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.319111 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.321858 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:04.321862 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:04.321864 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:04.321889 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:04.323616 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:04.324401 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:04.324407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:04.324409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:04.324545 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:04.325009 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:04.325012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:04.325013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:04.325212 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:04.325294 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:04.326050 (+ 756us) server_negotiation.cc:299] Negotiation successful
0810 07:23:04.326183 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 07:23:04.327813 25181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.312854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:46083)
0810 07:23:04.313608 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:04.313655 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:04.314525 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:04.315158 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:04.315166 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:04.315586 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:04.316127 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:04.316143 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.317461 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.317464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:04.318625 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:04.318636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.319164 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.319168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:04.319345 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.320060 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:04.320078 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:04.321679 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:04.323773 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:04.323784 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:04.323805 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:04.324236 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:04.324662 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:04.324669 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:04.324675 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:04.324902 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:04.325336 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:04.325346 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:04.325744 (+ 398us) client_negotiation.cc:769] Sending connection context
0810 07:23:04.326017 (+ 273us) client_negotiation.cc:240] Negotiation successful
0810 07:23:04.326488 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":142,"threads_started":1}
I20260810 07:23:04.329306 25178 heartbeater.cc:347] Connected to a master server at 127.1.125.254:34003
I20260810 07:23:04.329576 25178 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:04.330134 25178 heartbeater.cc:511] Master 127.1.125.254:34003 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:23:04.331769 24892 ts_manager.cc:194] Registered new tserver with Master: 8aa7968bd68a4b3eb023a62bcad8df51 (127.1.125.193:36935)
I20260810 07:23:04.333050 24892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:46083
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:23:04.351727 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:34003
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:40763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:04.469805 25184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:04.470131 25184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:04.470229 25184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:04.474562 25184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:04.474639 25184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:04.474762 25184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:23:04.480381 25184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:40763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:23:04.481585 25184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:04.482702 25184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:04.493074 25190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:04.493135 25191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:04.493245 25184 server_base.cc:1034] running on GCE node
W20260810 07:23:04.493331 25193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:04.493757 25184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:04.494438 25184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:04.495636 25184 hybrid_clock.cc:648] HybridClock initialized: now 1786346584495619 us; error 30 us; skew 500 ppm
Aug 10 07:23:04 dist-test-slave-44d2 krb5kdc[24820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346584, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:04.498431 25184 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:23:04.499317 25184 webserver.cc:460] Webserver started at http://127.1.125.194:43025/ using document root <none> and password file <none>
I20260810 07:23:04.499918 25184 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:04.500039 25184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:04.500275 25184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:04.502398 25184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "df52e2524f064df895a02ed44c01f577"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "c76f831350aed5ad7be217c8e7563dd1"
server_key_iv: "149f14e0e8452f63afb462f52de2ccbe"
server_key_version: "encryptionkey@0"
I20260810 07:23:04.502975 25184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "df52e2524f064df895a02ed44c01f577"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "c76f831350aed5ad7be217c8e7563dd1"
server_key_iv: "149f14e0e8452f63afb462f52de2ccbe"
server_key_version: "encryptionkey@0"
I20260810 07:23:04.506687 25184 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:23:04.509080 25204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:04.510303 25184 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:23:04.510440 25184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "df52e2524f064df895a02ed44c01f577"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "c76f831350aed5ad7be217c8e7563dd1"
server_key_iv: "149f14e0e8452f63afb462f52de2ccbe"
server_key_version: "encryptionkey@0"
I20260810 07:23:04.510551 25184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:04.520606 25184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:04.523715 25184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:04.523945 25184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:04.524686 25184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:04.525597 25184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:04.525646 25184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:04.525712 25184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:04.525751 25184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:04.535076 25184 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:44919
I20260810 07:23:04.535109 25367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:44919 every 8 connection(s)
I20260810 07:23:04.535967 25184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:23:04.539383 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 25184
I20260810 07:23:04.539521 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:23:04.539827 1527 external_mini_cluster.cc:1428] Setting key ed45a9397a84ff8751c83de2cd7c17fb
Aug 10 07:23:04 dist-test-slave-44d2 krb5kdc[24820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346584, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:23:04.550144 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.537820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:58925 (local address 127.1.125.254:34003)
0810 07:23:04.537973 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:04.537977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:04.539059 (+ 1082us) server_negotiation.cc:407] Connection header received
0810 07:23:04.540115 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:04.540119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:04.540179 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:04.540292 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:04.541666 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.542202 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.543368 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.543596 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.543706 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.546439 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:04.546443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:04.546445 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:04.546470 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:04.547977 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:04.548539 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:04.548545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:04.548546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:04.548638 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:04.549012 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:04.549016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:04.549017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:04.549197 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:04.549307 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:04.549824 (+ 517us) server_negotiation.cc:299] Negotiation successful
0810 07:23:04.549956 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
I20260810 07:23:04.550948 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.538218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:58925)
0810 07:23:04.538906 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:04.538947 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:04.539869 (+ 922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:04.540531 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:04.540542 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:04.540928 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:04.541523 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:04.541538 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.542332 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.542336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:04.543227 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:04.543237 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.543760 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.543764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:04.543940 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.544593 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:04.544610 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:04.546263 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:04.548116 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:04.548122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:04.548138 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:04.548399 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:04.548739 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:04.548743 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:04.548744 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:04.548889 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:04.549308 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:04.549315 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:04.549568 (+ 253us) client_negotiation.cc:769] Sending connection context
0810 07:23:04.549766 (+ 198us) client_negotiation.cc:240] Negotiation successful
0810 07:23:04.550097 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":118,"threads_started":1}
I20260810 07:23:04.552187 25368 heartbeater.cc:347] Connected to a master server at 127.1.125.254:34003
I20260810 07:23:04.552450 25368 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:04.553014 25368 heartbeater.cc:511] Master 127.1.125.254:34003 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
I20260810 07:23:04.554255 24892 ts_manager.cc:194] Registered new tserver with Master: df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919)
I20260810 07:23:04.554787 24892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:58925
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:23:04.571547 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:34003
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:40763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:04.692828 25381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:04.693117 25381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:04.693192 25381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:04.697206 25381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:04.697283 25381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:04.697365 25381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:23:04.702605 25381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:40763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:23:04.703677 25381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:04.704743 25381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:04.714504 25389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:04.714556 25387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:04.714757 25381 server_base.cc:1034] running on GCE node
W20260810 07:23:04.714929 25386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:04.715348 25381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:04.715962 25381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:04.717126 25381 hybrid_clock.cc:648] HybridClock initialized: now 1786346584717111 us; error 27 us; skew 500 ppm
Aug 10 07:23:04 dist-test-slave-44d2 krb5kdc[24820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346584, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:04.719920 25381 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:23:04.720840 25381 webserver.cc:460] Webserver started at http://127.1.125.195:45763/ using document root <none> and password file <none>
I20260810 07:23:04.721439 25381 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:04.721545 25381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:04.721783 25381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:23:04.724005 25381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "9072591c0a32473cad10cb1d00abb965"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "23db91b6c22a4848a75b4008b7298a4a"
server_key_iv: "8bc1aa380678549c2ece4ae5ee070f1d"
server_key_version: "encryptionkey@0"
I20260810 07:23:04.724680 25381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "9072591c0a32473cad10cb1d00abb965"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "23db91b6c22a4848a75b4008b7298a4a"
server_key_iv: "8bc1aa380678549c2ece4ae5ee070f1d"
server_key_version: "encryptionkey@0"
I20260810 07:23:04.728542 25381 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 07:23:04.730911 25398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:04.732087 25381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:23:04.732213 25381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "9072591c0a32473cad10cb1d00abb965"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "23db91b6c22a4848a75b4008b7298a4a"
server_key_iv: "8bc1aa380678549c2ece4ae5ee070f1d"
server_key_version: "encryptionkey@0"
I20260810 07:23:04.732316 25381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:04.749078 25381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:04.751770 25381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:04.751920 25381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:04.752460 25381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:04.753340 25381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:04.753387 25381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:04.753427 25381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:04.753448 25381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:04.763258 25381 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:39871
I20260810 07:23:04.763331 25563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:39871 every 8 connection(s)
I20260810 07:23:04.764195 25381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:23:04.769174 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 25381
I20260810 07:23:04.769279 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:23:04.769575 1527 external_mini_cluster.cc:1428] Setting key 09f1bb9ce80062628d716a229d03a060
Aug 10 07:23:04 dist-test-slave-44d2 krb5kdc[24820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346584, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:23:04.778172 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.766143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:52385 (local address 127.1.125.254:34003)
0810 07:23:04.766302 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:04.766308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:04.767469 (+ 1161us) server_negotiation.cc:407] Connection header received
0810 07:23:04.768428 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:04.768431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:04.768501 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:04.768559 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:04.769964 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.770630 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.771967 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.772193 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.772276 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.774900 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:04.774903 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:04.774905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:04.774932 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:04.776292 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:04.776843 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:04.776846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:04.776848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:04.776929 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:04.777240 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:04.777243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:04.777244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:04.777416 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:04.777493 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:04.777939 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 07:23:04.778039 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:23:04.779016 25566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.766588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:52385)
0810 07:23:04.767327 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:04.767371 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:04.768144 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:04.768770 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:04.768777 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:04.769169 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:04.769804 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:04.769821 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.770770 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.770773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:04.771861 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:04.771871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.772325 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.772328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:04.772465 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.773202 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:04.773219 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:04.774740 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:04.776402 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:04.776406 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:04.776418 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:04.776749 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:04.777016 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:04.777018 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:04.777019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:04.777160 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:04.777509 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:04.777514 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:04.777737 (+ 223us) client_negotiation.cc:769] Sending connection context
0810 07:23:04.777890 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 07:23:04.778149 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":119,"threads_started":1}
I20260810 07:23:04.780177 25564 heartbeater.cc:347] Connected to a master server at 127.1.125.254:34003
I20260810 07:23:04.780398 25564 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:04.780926 25564 heartbeater.cc:511] Master 127.1.125.254:34003 requested a full tablet report, sending...
I20260810 07:23:04.782007 24892 ts_manager.cc:194] Registered new tserver with Master: 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871)
I20260810 07:23:04.782752 24892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:52385
I20260810 07:23:04.783648 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:23:04.793180 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.785738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48000 (local address 127.1.125.254:34003)
0810 07:23:04.785886 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:04.785891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:04.786092 (+ 201us) server_negotiation.cc:407] Connection header received
0810 07:23:04.786235 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:04.786239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:04.786288 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:04.786372 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:04.787092 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.787587 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.788438 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.788722 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.788830 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.790165 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:04.790168 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:04.790171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:04.790199 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:04.791371 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:04.791806 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:04.791810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:04.791812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:04.791897 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:04.792185 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:04.792190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:04.792191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:04.792418 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:04.792629 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:04.792887 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 07:23:04.793005 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:23:04.795852 24892 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:23:04.798928 24892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:23:04.813848 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.807897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:39871 (local address 127.0.0.1:56010)
0810 07:23:04.808587 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:04.808603 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:04.808702 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:04.809242 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:04.809246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:04.809274 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:04.809449 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:04.809456 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.810767 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.810770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:04.812007 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:04.812016 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.813200 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.813203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:04.813319 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.813360 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 07:23:04.813499 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 07:23:04.813626 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":69,"spinlock_wait_cycles":2816,"thread_start_us":111,"threads_started":1}
I20260810 07:23:04.814595 25588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.808326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56010 (local address 127.1.125.195:39871)
0810 07:23:04.808782 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:04.808795 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:04.808824 (+ 29us) server_negotiation.cc:407] Connection header received
0810 07:23:04.808886 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:04.808890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:04.809108 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:04.809275 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:04.809577 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.810661 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.812151 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.813078 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.813188 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.813342 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:04.813539 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 07:23:04.813633 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":101,"threads_started":1}
I20260810 07:23:04.814746 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.807187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:44919 (local address 127.0.0.1:36890)
0810 07:23:04.808452 (+ 1265us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:04.808467 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:04.808582 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:04.809242 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:04.809246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:04.809277 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:04.809449 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:04.809456 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.811851 (+ 2395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.811855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:04.812811 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:04.812821 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.814201 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.814205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:04.814322 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.814413 (+ 91us) client_negotiation.cc:769] Sending connection context
0810 07:23:04.814483 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:23:04.814610 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1207,"mutex_wait_us":133,"spinlock_wait_cycles":13696,"thread_start_us":96,"threads_started":1}
I20260810 07:23:04.814827 25587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.807969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36890 (local address 127.1.125.194:44919)
0810 07:23:04.808392 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:04.808407 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:04.808684 (+ 277us) server_negotiation.cc:407] Connection header received
0810 07:23:04.808764 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:04.808768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:04.808934 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:04.809066 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:04.810830 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.811717 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.812939 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.813858 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.813982 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.814128 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:04.814527 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 07:23:04.814665 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":103,"threads_started":1}
I20260810 07:23:04.814978 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.807492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55634 (local address 127.1.125.193:36935)
0810 07:23:04.808432 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:04.808447 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:04.808474 (+ 27us) server_negotiation.cc:407] Connection header received
0810 07:23:04.808610 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:04.808618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:04.808827 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:04.808993 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:04.809313 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.810526 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.813042 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.813942 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.814035 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.814167 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:04.814630 (+ 463us) server_negotiation.cc:299] Negotiation successful
0810 07:23:04.814732 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":832,"thread_start_us":119,"threads_started":1}
I20260810 07:23:04.815099 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.807157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:36935 (local address 127.0.0.1:55634)
0810 07:23:04.808012 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:04.808065 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:04.808207 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:04.809000 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:04.809006 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:04.809031 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:04.809168 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:04.809178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.810665 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.810669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:04.812906 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:04.812919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:04.814217 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:04.814220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:04.814325 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:04.814406 (+ 81us) client_negotiation.cc:769] Sending connection context
0810 07:23:04.814483 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 07:23:04.814962 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":1343,"thread_start_us":80,"threads_started":1}
I20260810 07:23:04.817828 25467 tablet_service.cc:1467] Processing CreateTablet for tablet 6c51a4193ff04b29a0b270f75048db3e (DEFAULT_TABLE table=test-table [id=a4ddfa0065a84bdfa3736c8d916c34a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:04.817884 25267 tablet_service.cc:1467] Processing CreateTablet for tablet 6c51a4193ff04b29a0b270f75048db3e (DEFAULT_TABLE table=test-table [id=a4ddfa0065a84bdfa3736c8d916c34a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:04.817883 25075 tablet_service.cc:1467] Processing CreateTablet for tablet 6c51a4193ff04b29a0b270f75048db3e (DEFAULT_TABLE table=test-table [id=a4ddfa0065a84bdfa3736c8d916c34a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:04.818946 25467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c51a4193ff04b29a0b270f75048db3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:04.818969 25075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c51a4193ff04b29a0b270f75048db3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:04.818969 25267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c51a4193ff04b29a0b270f75048db3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:04.825390 25589 tablet_bootstrap.cc:492] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965: Bootstrap starting.
I20260810 07:23:04.826790 25592 tablet_bootstrap.cc:492] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577: Bootstrap starting.
I20260810 07:23:04.827132 25590 tablet_bootstrap.cc:492] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Bootstrap starting.
I20260810 07:23:04.827945 25589 tablet_bootstrap.cc:654] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:04.829290 25590 tablet_bootstrap.cc:654] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:04.829447 25589 log.cc:824] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:04.829674 25592 tablet_bootstrap.cc:654] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:04.830192 25590 log.cc:824] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:04.830404 25592 log.cc:824] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:04.831600 25589 tablet_bootstrap.cc:492] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965: No bootstrap required, opened a new log
I20260810 07:23:04.831823 25589 ts_tablet_manager.cc:1397] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260810 07:23:04.832345 25592 tablet_bootstrap.cc:492] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577: No bootstrap required, opened a new log
I20260810 07:23:04.832345 25590 tablet_bootstrap.cc:492] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: No bootstrap required, opened a new log
I20260810 07:23:04.832561 25590 ts_tablet_manager.cc:1397] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:23:04.832561 25592 ts_tablet_manager.cc:1397] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:23:04.834782 25589 raft_consensus.cc:348] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } }
I20260810 07:23:04.835064 25589 raft_consensus.cc:374] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:04.835121 25589 raft_consensus.cc:729] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9072591c0a32473cad10cb1d00abb965, State: Initialized, Role: FOLLOWER
I20260810 07:23:04.835836 25589 consensus_queue.cc:260] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } }
I20260810 07:23:04.836825 25564 heartbeater.cc:503] Master 127.1.125.254:34003 was elected leader, sending a full tablet report...
I20260810 07:23:04.836997 25592 raft_consensus.cc:348] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } }
I20260810 07:23:04.837313 25592 raft_consensus.cc:374] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:04.837147 25590 raft_consensus.cc:348] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } }
I20260810 07:23:04.837430 25590 raft_consensus.cc:374] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:04.837431 25592 raft_consensus.cc:729] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df52e2524f064df895a02ed44c01f577, State: Initialized, Role: FOLLOWER
I20260810 07:23:04.837508 25590 raft_consensus.cc:729] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aa7968bd68a4b3eb023a62bcad8df51, State: Initialized, Role: FOLLOWER
I20260810 07:23:04.837949 25592 consensus_queue.cc:260] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } }
I20260810 07:23:04.837949 25590 consensus_queue.cc:260] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } }
I20260810 07:23:04.838039 25589 ts_tablet_manager.cc:1428] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260810 07:23:04.839004 25592 ts_tablet_manager.cc:1428] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:23:04.839064 25178 heartbeater.cc:503] Master 127.1.125.254:34003 was elected leader, sending a full tablet report...
I20260810 07:23:04.839111 25368 heartbeater.cc:503] Master 127.1.125.254:34003 was elected leader, sending a full tablet report...
I20260810 07:23:04.839990 25590 ts_tablet_manager.cc:1428] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 07:23:04.998270 25603 raft_consensus.cc:482] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:04.998543 25603 raft_consensus.cc:504] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } }
I20260810 07:23:04.999763 25603 leader_election.cc:288] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8aa7968bd68a4b3eb023a62bcad8df51 (127.1.125.193:36935), 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871)
I20260810 07:23:05.004180 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:04.999977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:36935 (local address 127.1.125.194:57155)
0810 07:23:05.000164 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:05.000183 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:05.000283 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:05.000751 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:05.000754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:05.000773 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:05.000893 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:05.000899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:05.001848 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:05.001853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:05.002838 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:05.002849 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:05.003736 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:05.003739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:05.003856 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:05.003882 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:23:05.003961 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:23:05.004041 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 07:23:05.004251 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:05.000118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:57155 (local address 127.1.125.193:36935)
0810 07:23:05.000271 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:05.000276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:05.000295 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:23:05.000447 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:05.000451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:05.000551 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:05.000627 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:05.001032 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:05.001736 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:05.002981 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:05.003608 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:05.003749 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:05.003813 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:05.003996 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 07:23:05.004078 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 07:23:05.004911 25588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:05.000575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:39035 (local address 127.1.125.195:39871)
0810 07:23:05.000709 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:05.000714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:05.000864 (+ 150us) server_negotiation.cc:407] Connection header received
0810 07:23:05.001123 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:05.001127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:05.001179 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:05.001251 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:05.001553 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:05.002296 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:05.003531 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:05.004108 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:05.004228 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:05.004296 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:05.004727 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 07:23:05.004819 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:23:05.004865 25103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c51a4193ff04b29a0b270f75048db3e" candidate_uuid: "df52e2524f064df895a02ed44c01f577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" is_pre_election: true
I20260810 07:23:05.005168 25103 raft_consensus.cc:2454] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df52e2524f064df895a02ed44c01f577 in term 0.
I20260810 07:23:05.005306 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:05.000257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:39871 (local address 127.1.125.194:39035)
0810 07:23:05.000748 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:05.000769 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:05.000924 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:05.001289 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:05.001292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:05.001311 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:05.001429 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:05.001435 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:05.002427 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:05.002430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:05.003406 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:05.003416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:05.004416 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:05.004423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:05.004590 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:05.004613 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:23:05.005079 (+ 466us) client_negotiation.cc:240] Negotiation successful
0810 07:23:05.005160 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":167,"threads_started":1}
I20260810 07:23:05.005718 25208 leader_election.cc:302] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8aa7968bd68a4b3eb023a62bcad8df51, df52e2524f064df895a02ed44c01f577; no voters:
I20260810 07:23:05.006026 25603 raft_consensus.cc:2781] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:23:05.006007 25492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c51a4193ff04b29a0b270f75048db3e" candidate_uuid: "df52e2524f064df895a02ed44c01f577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9072591c0a32473cad10cb1d00abb965" is_pre_election: true
I20260810 07:23:05.006139 25603 raft_consensus.cc:482] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:23:05.006194 25603 raft_consensus.cc:3037] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:05.006266 25492 raft_consensus.cc:2454] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df52e2524f064df895a02ed44c01f577 in term 0.
I20260810 07:23:05.007434 25603 raft_consensus.cc:504] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } }
I20260810 07:23:05.007793 25603 leader_election.cc:288] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [CANDIDATE]: Term 1 election: Requested vote from peers 8aa7968bd68a4b3eb023a62bcad8df51 (127.1.125.193:36935), 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871)
I20260810 07:23:05.008219 25103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c51a4193ff04b29a0b270f75048db3e" candidate_uuid: "df52e2524f064df895a02ed44c01f577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aa7968bd68a4b3eb023a62bcad8df51"
I20260810 07:23:05.008230 25492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c51a4193ff04b29a0b270f75048db3e" candidate_uuid: "df52e2524f064df895a02ed44c01f577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9072591c0a32473cad10cb1d00abb965"
I20260810 07:23:05.008347 25103 raft_consensus.cc:3037] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:05.008369 25492 raft_consensus.cc:3037] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:05.009444 25492 raft_consensus.cc:2454] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df52e2524f064df895a02ed44c01f577 in term 1.
I20260810 07:23:05.009733 25103 raft_consensus.cc:2454] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df52e2524f064df895a02ed44c01f577 in term 1.
I20260810 07:23:05.009760 25208 leader_election.cc:302] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9072591c0a32473cad10cb1d00abb965, df52e2524f064df895a02ed44c01f577; no voters:
I20260810 07:23:05.009948 25603 raft_consensus.cc:2781] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:05.010187 25603 raft_consensus.cc:686] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [term 1 LEADER]: Becoming Leader. State: Replica: df52e2524f064df895a02ed44c01f577, State: Running, Role: LEADER
I20260810 07:23:05.010526 25603 consensus_queue.cc:237] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } }
I20260810 07:23:05.013867 24890 catalog_manager.cc:5495] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 reported cstate change: term changed from 0 to 1, leader changed from <none> to df52e2524f064df895a02ed44c01f577 (127.1.125.194). New cstate: current_term: 1 leader_uuid: "df52e2524f064df895a02ed44c01f577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } health_report { overall_health: UNKNOWN } } }
W20260810 07:23:05.015169 25565 tablet.cc:2369] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:23:05.037137 25369 tablet.cc:2369] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:23:05.047078 25587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:05.043869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36896 (local address 127.1.125.194:44919)
0810 07:23:05.044013 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:05.044017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:05.044029 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:23:05.044101 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:05.044103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:05.044157 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:05.044248 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:23:05.044509 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:05.045012 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:05.045814 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:05.046027 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:05.046101 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:05.046355 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:23:05.046751 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:23:05.046840 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:05.046898 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 07:23:05.046947 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:23:05.052990 25492 raft_consensus.cc:1261] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965 [term 1 FOLLOWER]: Refusing update from remote peer df52e2524f064df895a02ed44c01f577: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:05.053000 25103 raft_consensus.cc:1261] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Refusing update from remote peer df52e2524f064df895a02ed44c01f577: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:05.053733 25603 consensus_queue.cc:1035] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:05.053918 25606 consensus_queue.cc:1035] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:05.060689 25613 mvcc.cc:205] Tried to move back new op lower bound from 7316875612369625088 to 7316875612206186496. Current Snapshot: MvccSnapshot[applied={T|T < 7316875612369625088}]
I20260810 07:23:05.062853 25615 mvcc.cc:205] Tried to move back new op lower bound from 7316875612369625088 to 7316875612206186496. Current Snapshot: MvccSnapshot[applied={T|T < 7316875612369625088}]
I20260810 07:23:05.063300 25616 mvcc.cc:205] Tried to move back new op lower bound from 7316875612369625088 to 7316875612206186496. Current Snapshot: MvccSnapshot[applied={T|T < 7316875612369625088}]
I20260810 07:23:05.078246 24890 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:05.078606 24890 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:05.083366 24890 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Sending DeleteTablet for 3 replicas of tablet 6c51a4193ff04b29a0b270f75048db3e
I20260810 07:23:05.085549 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 24977
I20260810 07:23:05.085839 25467 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c51a4193ff04b29a0b270f75048db3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:05 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:56010
I20260810 07:23:05.085839 25267 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c51a4193ff04b29a0b270f75048db3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:05 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:36890
I20260810 07:23:05.114207 25630 tablet_replica.cc:330] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965: stopping tablet replica
I20260810 07:23:05.115119 25631 tablet_replica.cc:330] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577: stopping tablet replica
I20260810 07:23:05.115356 25630 raft_consensus.cc:2229] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:05.115887 25631 raft_consensus.cc:2229] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:23:05.116371 25630 raft_consensus.cc:2258] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:05.116950 25631 raft_consensus.cc:2258] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:05.120700 25631 ts_tablet_manager.cc:1905] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:05.121009 25630 ts_tablet_manager.cc:1905] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:05.128963 25631 ts_tablet_manager.cc:1918] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:05.129108 25630 ts_tablet_manager.cc:1918] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:05.129189 25631 log.cc:1196] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/6c51a4193ff04b29a0b270f75048db3e
I20260810 07:23:05.129336 25630 log.cc:1196] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/6c51a4193ff04b29a0b270f75048db3e
I20260810 07:23:05.130044 25631 ts_tablet_manager.cc:1939] T 6c51a4193ff04b29a0b270f75048db3e P df52e2524f064df895a02ed44c01f577: Deleting consensus metadata
I20260810 07:23:05.130532 25630 ts_tablet_manager.cc:1939] T 6c51a4193ff04b29a0b270f75048db3e P 9072591c0a32473cad10cb1d00abb965: Deleting consensus metadata
I20260810 07:23:05.133491 24872 catalog_manager.cc:4841] TS df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919): tablet 6c51a4193ff04b29a0b270f75048db3e (table test-table [id=a4ddfa0065a84bdfa3736c8d916c34a3]) successfully deleted
W20260810 07:23:05.133829 24871 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:36935 (error 108)
I20260810 07:23:05.134790 24871 catalog_manager.cc:4841] TS 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871): tablet 6c51a4193ff04b29a0b270f75048db3e (table test-table [id=a4ddfa0065a84bdfa3736c8d916c34a3]) successfully deleted
I20260810 07:23:05.135751 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 25184
I20260810 07:23:05.136199 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:05.135358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:36935 (local address 127.0.0.1:55640)
0810 07:23:05.135792 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:05.135976 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:36935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
W20260810 07:23:05.136579 24871 catalog_manager.cc:4584] TS 8aa7968bd68a4b3eb023a62bcad8df51 (127.1.125.193:36935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c51a4193ff04b29a0b270f75048db3e: Network error: Client connection negotiation failed: client connection to 127.1.125.193:36935: connect: Connection refused (error 111)
I20260810 07:23:05.175643 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 25381
I20260810 07:23:05.196234 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 24842
I20260810 07:23:25.219223 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:34003
--webserver_interface=127.1.125.254
--webserver_port=42147
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:40763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:34003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:25.336954 25634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:25.337218 25634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:25.337260 25634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:25.341303 25634 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 07:23:25.341365 25634 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 07:23:25.341382 25634 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 07:23:25.341395 25634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:25.341464 25634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:25.341496 25634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:25.341537 25634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:25.341588 25634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:25.347554 25634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:40763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:34003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:34003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=42147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:23:25.348733 25634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:25.349936 25634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:23:25.364970 25634 server_base.cc:1034] running on GCE node
W20260810 07:23:25.364908 25647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:25.365026 25645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:25.364908 25643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:25.365932 25634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:25.367185 25634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:25.368363 25634 hybrid_clock.cc:648] HybridClock initialized: now 1786346605368346 us; error 28 us; skew 500 ppm
Aug 10 07:23:25 dist-test-slave-44d2 krb5kdc[24820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346605, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:25.371297 25634 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:23:25.372192 25634 webserver.cc:460] Webserver started at http://127.1.125.254:42147/ using document root <none> and password file <none>
I20260810 07:23:25.372852 25634 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:25.372934 25634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:25.376938 25634 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:23:25.379252 25660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:25.380743 25634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:23:25.380901 25634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "4978926106d341a6a9962256304bc518"
format_stamp: "Formatted at 2026-08-10 07:23:03 on dist-test-slave-44d2"
server_key: "5a594dc1fe586cbbdf2fff07c816a52b"
server_key_iv: "42513ba2ffab905ed844d0351985cf73"
server_key_version: "encryptionkey@0"
I20260810 07:23:25.381562 25634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:25.392680 25634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:25.395548 25634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:25.395836 25634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:25.404075 25634 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:34003
I20260810 07:23:25.404362 25745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:34003 every 8 connection(s)
I20260810 07:23:25.405304 25634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:25.407681 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 25634
I20260810 07:23:25.408146 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:36935
--local_ip_for_outbound_sockets=127.1.125.193
--tserver_master_addrs=127.1.125.254:34003
--webserver_port=44927
--webserver_interface=127.1.125.193
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:40763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:23:25.410174 25747 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:23:25.412642 25747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Bootstrap starting.
I20260810 07:23:25.430680 25747 log.cc:824] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:25.439442 25747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:23:25.439844 25747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Bootstrap complete.
I20260810 07:23:25.443949 25747 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } }
I20260810 07:23:25.444336 25747 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4978926106d341a6a9962256304bc518, State: Initialized, Role: FOLLOWER
I20260810 07:23:25.444844 25747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } }
I20260810 07:23:25.444957 25747 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:25.445060 25747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:25.445152 25747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:23:25.447079 25747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } }
I20260810 07:23:25.447415 25747 leader_election.cc:302] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4978926106d341a6a9962256304bc518; no voters:
I20260810 07:23:25.447763 25747 leader_election.cc:288] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:23:25.447846 25759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:23:25.448045 25759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 2 LEADER]: Becoming Leader. State: Replica: 4978926106d341a6a9962256304bc518, State: Running, Role: LEADER
I20260810 07:23:25.448339 25759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } }
I20260810 07:23:25.449070 25747 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:25.449308 25759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4978926106d341a6a9962256304bc518. Latest consensus state: current_term: 2 leader_uuid: "4978926106d341a6a9962256304bc518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } } }
I20260810 07:23:25.449419 25759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:25.449854 25764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:25.449869 25760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4978926106d341a6a9962256304bc518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } } }
I20260810 07:23:25.450055 25760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:25.454717 25764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:25.455336 25764 catalog_manager.cc:1259] Loaded cluster ID: 179fd7238e484fe7b99425dc87612281
I20260810 07:23:25.455453 25764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:25.458262 25764 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:25.459031 25764 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Loaded TSK: 0
I20260810 07:23:25.472985 25764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Generated new TSK 1
I20260810 07:23:25.473265 25764 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:23:25.534220 25756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:25.534475 25756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:25.534518 25756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:25.538559 25756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:25.538621 25756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:25.538681 25756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:23:25.543924 25756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:40763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:36935
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=44927
--tserver_master_addrs=127.1.125.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:23:25.545007 25756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:25.546042 25756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:25.556173 25797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:25.556190 25796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:25.556213 25799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:25.556665 25756 server_base.cc:1034] running on GCE node
I20260810 07:23:25.557140 25756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:25.557889 25756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:25.559092 25756 hybrid_clock.cc:648] HybridClock initialized: now 1786346605559070 us; error 41 us; skew 500 ppm
Aug 10 07:23:25 dist-test-slave-44d2 krb5kdc[24820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346605, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:25.562201 25756 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:23:25.563216 25756 webserver.cc:460] Webserver started at http://127.1.125.193:44927/ using document root <none> and password file <none>
I20260810 07:23:25.563825 25756 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:25.563901 25756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:25.567975 25756 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:23:25.570297 25809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:25.571601 25756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:23:25.571699 25756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "8aa7968bd68a4b3eb023a62bcad8df51"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "13fff1810b5ef56d60e97c9428535813"
server_key_iv: "d0d7ac9b704b4026d73ef4bee48e6754"
server_key_version: "encryptionkey@0"
I20260810 07:23:25.572257 25756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:25.597795 25756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:25.600957 25756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:25.601191 25756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:25.601857 25756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:25.603111 25818 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:23:25.607435 25756 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:23:25.607496 25756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260810 07:23:25.607573 25756 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:23:25.609364 25756 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:23:25.609411 25756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260810 07:23:25.609689 25818 tablet_bootstrap.cc:492] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Bootstrap starting.
I20260810 07:23:25.621276 25756 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:36935
I20260810 07:23:25.621366 25977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:36935 every 8 connection(s)
I20260810 07:23:25.622375 25756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:23:25.627205 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 25756
I20260810 07:23:25.627748 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:44919
--local_ip_for_outbound_sockets=127.1.125.194
--tserver_master_addrs=127.1.125.254:34003
--webserver_port=43025
--webserver_interface=127.1.125.194
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:40763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:23:25.630208 25818 log.cc:824] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Log is configured to *not* fsync() on all Append() calls
Aug 10 07:23:25 dist-test-slave-44d2 krb5kdc[24820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346605, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:23:25.639189 25818 tablet_bootstrap.cc:492] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:23:25.639627 25818 tablet_bootstrap.cc:492] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Bootstrap complete.
I20260810 07:23:25.641371 25818 ts_tablet_manager.cc:1397] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Time spent bootstrapping tablet: real 0.032s user 0.016s sys 0.015s
I20260810 07:23:25.643299 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:25.624586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:52785 (local address 127.1.125.254:34003)
0810 07:23:25.625064 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:25.625090 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:25.625353 (+ 263us) server_negotiation.cc:407] Connection header received
0810 07:23:25.626539 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:25.626559 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:25.627015 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:25.627450 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:25.629307 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:25.630780 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:25.632415 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:25.633353 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:25.633570 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:25.636916 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:25.636923 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:25.636939 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:25.636974 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:25.639298 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:25.640061 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:25.640070 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:25.640078 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:25.640242 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:25.640722 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:25.640728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:25.640733 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:25.641241 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:25.641403 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:25.642117 (+ 714us) server_negotiation.cc:299] Negotiation successful
0810 07:23:25.642454 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":130,"threads_started":1}
I20260810 07:23:25.643342 25983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:25.624449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:52785)
0810 07:23:25.625164 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:25.625220 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:25.626053 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:25.627547 (+ 1494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:25.627556 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:25.628306 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:25.629082 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:25.629107 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:25.630989 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:25.630992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:25.632266 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:25.632278 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:25.633524 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:25.633528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:25.633724 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:25.634480 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:25.634502 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:25.636654 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:25.639443 (+ 2789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:25.639452 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:25.639469 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:25.639899 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:25.640361 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:25.640364 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:25.640367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:25.640602 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:25.641386 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:25.641396 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:25.641729 (+ 333us) client_negotiation.cc:769] Sending connection context
0810 07:23:25.641940 (+ 211us) client_negotiation.cc:240] Negotiation successful
0810 07:23:25.642285 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":161,"threads_started":1}
I20260810 07:23:25.644517 25818 raft_consensus.cc:348] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } }
I20260810 07:23:25.645054 25818 raft_consensus.cc:729] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aa7968bd68a4b3eb023a62bcad8df51, State: Initialized, Role: FOLLOWER
I20260810 07:23:25.645567 25818 consensus_queue.cc:260] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } }
I20260810 07:23:25.646548 25978 heartbeater.cc:347] Connected to a master server at 127.1.125.254:34003
I20260810 07:23:25.646750 25818 ts_tablet_manager.cc:1428] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260810 07:23:25.646806 25978 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:25.647285 25978 heartbeater.cc:511] Master 127.1.125.254:34003 requested a full tablet report, sending...
I20260810 07:23:25.649835 25685 ts_manager.cc:194] Registered new tserver with Master: 8aa7968bd68a4b3eb023a62bcad8df51 (127.1.125.193:36935)
I20260810 07:23:25.653394 25685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:52785
I20260810 07:23:25.655582 25978 heartbeater.cc:503] Master 127.1.125.254:34003 was elected leader, sending a full tablet report...
I20260810 07:23:25.660259 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:25.652845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:36935 (local address 127.0.0.1:40162)
0810 07:23:25.656054 (+ 3209us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:25.656103 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:25.656256 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:25.656873 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:25.656879 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:25.656901 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:25.657150 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:25.657158 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:25.658101 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:25.658106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:25.658959 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:25.658968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:25.659720 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:25.659724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:25.659833 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:25.659872 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 07:23:25.660003 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 07:23:25.660127 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3021,"thread_start_us":135,"threads_started":1}
I20260810 07:23:25.660259 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:25.652908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40162 (local address 127.1.125.193:36935)
0810 07:23:25.653248 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:25.653258 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:25.656394 (+ 3136us) server_negotiation.cc:407] Connection header received
0810 07:23:25.656466 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:25.656471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:25.656701 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:25.656859 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:25.657274 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:25.657982 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:25.659092 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:25.659593 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:25.659720 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:25.659853 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:25.660033 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 07:23:25.660120 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":96,"threads_started":1}
I20260810 07:23:25.661298 25878 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c51a4193ff04b29a0b270f75048db3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:05 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:40162
I20260810 07:23:25.662294 25990 tablet_replica.cc:330] stopping tablet replica
I20260810 07:23:25.662606 25990 raft_consensus.cc:2229] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:25.662856 25990 raft_consensus.cc:2258] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:25.663844 25990 ts_tablet_manager.cc:1905] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:25.667397 25990 ts_tablet_manager.cc:1918] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:25.667527 25990 log.cc:1196] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal/wals/6c51a4193ff04b29a0b270f75048db3e
I20260810 07:23:25.667798 25990 ts_tablet_manager.cc:1939] T 6c51a4193ff04b29a0b270f75048db3e P 8aa7968bd68a4b3eb023a62bcad8df51: Deleting consensus metadata
I20260810 07:23:25.669552 25666 catalog_manager.cc:4841] TS 8aa7968bd68a4b3eb023a62bcad8df51 (127.1.125.193:36935): tablet 6c51a4193ff04b29a0b270f75048db3e (table test-table [id=a4ddfa0065a84bdfa3736c8d916c34a3]) successfully deleted
W20260810 07:23:25.752737 25985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:25.752991 25985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:25.753034 25985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:25.757103 25985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:25.757160 25985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:25.757217 25985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:23:25.762346 25985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:40763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:44919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=43025
--tserver_master_addrs=127.1.125.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:23:25.763298 25985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:25.764338 25985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:25.774344 25997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:25.774514 26000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:25.774508 25985 server_base.cc:1034] running on GCE node
W20260810 07:23:25.774586 26002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:25.775085 25985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:25.775736 25985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:25.776934 25985 hybrid_clock.cc:648] HybridClock initialized: now 1786346605776900 us; error 45 us; skew 500 ppm
Aug 10 07:23:25 dist-test-slave-44d2 krb5kdc[24820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346605, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:25.779906 25985 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:23:25.780891 25985 webserver.cc:460] Webserver started at http://127.1.125.194:43025/ using document root <none> and password file <none>
I20260810 07:23:25.781428 25985 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:25.781502 25985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:25.785859 25985 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:23:25.788107 26011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:25.789510 25985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:23:25.789646 25985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "df52e2524f064df895a02ed44c01f577"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "c76f831350aed5ad7be217c8e7563dd1"
server_key_iv: "149f14e0e8452f63afb462f52de2ccbe"
server_key_version: "encryptionkey@0"
I20260810 07:23:25.790206 25985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:25.809152 25985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:25.812774 25985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:25.813006 25985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:25.813664 25985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:25.814559 25985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:25.814607 25985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:25.814678 25985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:25.814723 25985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:25.824075 25985 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:44919
I20260810 07:23:25.824107 26183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:44919 every 8 connection(s)
I20260810 07:23:25.825103 25985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:23:25.827225 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 25985
I20260810 07:23:25.827751 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:39871
--local_ip_for_outbound_sockets=127.1.125.195
--tserver_master_addrs=127.1.125.254:34003
--webserver_port=45763
--webserver_interface=127.1.125.195
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:40763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:23:25 dist-test-slave-44d2 krb5kdc[24820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346605, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:23:25.840082 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:25.826924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:52927 (local address 127.1.125.254:34003)
0810 07:23:25.827107 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:25.827111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:25.828368 (+ 1257us) server_negotiation.cc:407] Connection header received
0810 07:23:25.829361 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:25.829364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:25.829431 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:25.829544 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:25.830746 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:25.831397 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:25.832680 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:25.833129 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:25.833269 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:25.835930 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:25.835937 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:25.835941 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:25.835978 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:25.837827 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:25.838328 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:25.838334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:25.838336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:25.838438 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:25.838806 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:25.838810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:25.838812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:25.839088 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:25.839214 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:25.839689 (+ 475us) server_negotiation.cc:299] Negotiation successful
0810 07:23:25.839851 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 07:23:25.840862 26186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:25.827390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:52927)
0810 07:23:25.828217 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:25.828259 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:25.829135 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:25.829728 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:25.829735 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:25.830099 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:25.830576 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:25.830589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:25.831542 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:25.831545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:25.832467 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:25.832544 (+ 77us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:25.833387 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:25.833391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:25.833538 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:25.834235 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:25.834253 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:25.835742 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:25.837948 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:25.837953 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:25.837966 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:25.838220 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:25.838554 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:25.838556 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:25.838557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:25.838704 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:25.839200 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:25.839205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:25.839429 (+ 224us) client_negotiation.cc:769] Sending connection context
0810 07:23:25.839603 (+ 174us) client_negotiation.cc:240] Negotiation successful
0810 07:23:25.839863 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":152,"threads_started":1}
I20260810 07:23:25.842173 26184 heartbeater.cc:347] Connected to a master server at 127.1.125.254:34003
I20260810 07:23:25.842440 26184 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:25.843001 26184 heartbeater.cc:511] Master 127.1.125.254:34003 requested a full tablet report, sending...
I20260810 07:23:25.844118 25685 ts_manager.cc:194] Registered new tserver with Master: df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919)
I20260810 07:23:25.844805 25685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:52927
W20260810 07:23:25.948704 26187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:25.949029 26187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:25.949112 26187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:25.953496 26187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:25.953553 26187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:25.953655 26187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:23:25.958865 26187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:40763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:39871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=45763
--tserver_master_addrs=127.1.125.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:23:25.959888 26187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:25.960955 26187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:25.970810 26198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:25.970855 26201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:25.970907 26187 server_base.cc:1034] running on GCE node
W20260810 07:23:25.970835 26197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:25.971457 26187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:25.972100 26187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:25.973292 26187 hybrid_clock.cc:648] HybridClock initialized: now 1786346605973258 us; error 44 us; skew 500 ppm
Aug 10 07:23:25 dist-test-slave-44d2 krb5kdc[24820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346605, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:25.976246 26187 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:23:25.977195 26187 webserver.cc:460] Webserver started at http://127.1.125.195:45763/ using document root <none> and password file <none>
I20260810 07:23:25.977766 26187 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:25.977881 26187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:25.981736 26187 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:23:25.983788 26213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:25.985231 26187 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:23:25.985368 26187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "9072591c0a32473cad10cb1d00abb965"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "23db91b6c22a4848a75b4008b7298a4a"
server_key_iv: "8bc1aa380678549c2ece4ae5ee070f1d"
server_key_version: "encryptionkey@0"
I20260810 07:23:25.985975 26187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:26.003743 26187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:26.007124 26187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:26.007345 26187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:26.007961 26187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:26.008888 26187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:26.008963 26187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:26.009048 26187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:26.009102 26187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:26.018640 26187 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:39871
I20260810 07:23:26.018669 26383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:39871 every 8 connection(s)
I20260810 07:23:26.019644 26187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:23:26.026013 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 26187
Aug 10 07:23:26 dist-test-slave-44d2 krb5kdc[24820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346605, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:23:26.034016 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.021631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:55725 (local address 127.1.125.254:34003)
0810 07:23:26.021781 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:26.021785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:26.022835 (+ 1050us) server_negotiation.cc:407] Connection header received
0810 07:23:26.023659 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:26.023661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:26.023707 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:26.023783 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:26.024981 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.025496 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.026846 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.027207 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.027320 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.030350 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:26.030354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:26.030356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:26.030378 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:26.031932 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:26.032460 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:26.032469 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:26.032470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:26.032608 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:26.032950 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:26.032954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:26.032956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:26.033130 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:26.033237 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:26.033716 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 07:23:26.033834 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64,"spinlock_wait_cycles":896}
I20260810 07:23:26.035092 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.026717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50348 (local address 127.1.125.254:34003)
0810 07:23:26.027004 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:26.027008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:26.027184 (+ 176us) server_negotiation.cc:407] Connection header received
0810 07:23:26.027336 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:26.027339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:26.027385 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:26.027480 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:26.028353 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.029194 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.030254 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.030480 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.030591 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.031856 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:26.031860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:26.031862 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:26.031893 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:26.033445 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:26.033894 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:26.033897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:26.033899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:26.033983 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:26.034297 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:26.034300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:26.034302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:26.034514 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:26.034728 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:26.034835 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 07:23:26.034946 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":112,"threads_started":1}
I20260810 07:23:26.035123 26386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.022022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:55725)
0810 07:23:26.022688 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:26.022728 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:26.023469 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:26.023949 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:26.023956 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:26.024297 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:26.024856 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:26.024870 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.025606 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.025609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:26.026705 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:26.026717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.027391 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.027396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:26.027621 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.028279 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:26.028320 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:26.030180 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:26.032063 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:26.032069 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:26.032084 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:26.032344 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:26.032704 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:26.032707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:26.032709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:26.032851 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:26.033239 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:26.033246 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:26.033482 (+ 236us) client_negotiation.cc:769] Sending connection context
0810 07:23:26.033670 (+ 188us) client_negotiation.cc:240] Negotiation successful
0810 07:23:26.033973 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":120,"threads_started":1}
I20260810 07:23:26.036309 26384 heartbeater.cc:347] Connected to a master server at 127.1.125.254:34003
I20260810 07:23:26.036618 26384 heartbeater.cc:464] Registering TS with master...
I20260810 07:23:26.037186 26384 heartbeater.cc:511] Master 127.1.125.254:34003 requested a full tablet report, sending...
I20260810 07:23:26.038194 25685 ts_manager.cc:194] Registered new tserver with Master: 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871)
I20260810 07:23:26.038736 25685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:55725
I20260810 07:23:26.049001 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:23:26.053050 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.049628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50350 (local address 127.1.125.254:34003)
0810 07:23:26.049788 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:26.049791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:26.049924 (+ 133us) server_negotiation.cc:407] Connection header received
0810 07:23:26.050089 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:26.050092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:26.050237 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:26.050322 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:23:26.050537 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.051024 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.051944 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.052168 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.052246 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.052571 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:23:26.052717 (+ 146us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 07:23:26.052917 (+ 200us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50350: token expired
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 07:23:26.053128 26390 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50350: token expired
W20260810 07:23:26.053442 25571 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.125.254:34003)
I20260810 07:23:26.060282 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.053847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50354 (local address 127.1.125.254:34003)
0810 07:23:26.053958 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:26.053961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:26.053973 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:23:26.054053 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:26.054055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:26.054086 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:26.054147 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:26.054706 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.055161 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.055882 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.056082 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.056154 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.057539 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:26.057542 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:26.057544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:26.057562 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:26.058792 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:26.059197 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:26.059200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:26.059201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:26.059274 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:26.059504 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:26.059506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:26.059507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:26.059688 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:26.059790 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:26.059981 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 07:23:26.060076 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:23:26.062779 25685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:23:26.066129 25685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:23:26.074339 25878 tablet_service.cc:1467] Processing CreateTablet for tablet 9fc8d9031ba049389e7a0213cdeddf29 (DEFAULT_TABLE table=test-table [id=7bdb94438617462385e4a3b752826712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:26.075078 25878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fc8d9031ba049389e7a0213cdeddf29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:26.078135 25818 tablet_bootstrap.cc:492] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51: Bootstrap starting.
I20260810 07:23:26.078720 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.073056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:39871 (local address 127.0.0.1:41998)
0810 07:23:26.073203 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:26.073221 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:26.073306 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:26.074369 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:26.074372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:26.074391 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:26.074564 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:26.074575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.075937 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.075944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:26.077074 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:26.077085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.078279 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.078283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:26.078374 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.078416 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 07:23:26.078475 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:23:26.078556 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":40}
I20260810 07:23:26.079079 26397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.073319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:44919 (local address 127.0.0.1:54934)
0810 07:23:26.073585 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:26.073598 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:26.073690 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:26.074426 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:26.074429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:26.074449 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:26.074601 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:26.074608 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.076283 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.076287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:26.077402 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:26.077410 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.078665 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.078669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:26.078768 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.078798 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 07:23:26.078868 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:23:26.078932 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":5,"thread_start_us":95,"threads_started":1}
I20260810 07:23:26.079244 25818 tablet_bootstrap.cc:654] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:26.079530 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.073554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54934 (local address 127.1.125.194:44919)
0810 07:23:26.073906 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:26.073923 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:26.073948 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:23:26.074035 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:26.074040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:26.074245 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:26.074769 (+ 524us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:26.074995 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.076157 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.077540 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.078532 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.078664 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.079043 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:26.079107 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 07:23:26.079415 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":156,"threads_started":1}
I20260810 07:23:26.079568 26399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.073297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41998 (local address 127.1.125.195:39871)
0810 07:23:26.073955 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:26.073965 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:26.073991 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:23:26.074058 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:26.074062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:26.074224 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:26.074386 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:26.074705 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.075808 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.077213 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.078124 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.079012 (+ 888us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.079245 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:26.079308 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 07:23:26.079414 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":183,"threads_started":1}
I20260810 07:23:26.080387 25818 tablet_bootstrap.cc:492] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51: No bootstrap required, opened a new log
I20260810 07:23:26.080544 25818 ts_tablet_manager.cc:1397] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260810 07:23:26.080973 25818 raft_consensus.cc:348] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:26.081084 25818 raft_consensus.cc:374] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:26.081120 25818 raft_consensus.cc:729] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aa7968bd68a4b3eb023a62bcad8df51, State: Initialized, Role: FOLLOWER
I20260810 07:23:26.081243 25818 consensus_queue.cc:260] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:26.081502 25818 ts_tablet_manager.cc:1428] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:23:26.082794 26084 tablet_service.cc:1467] Processing CreateTablet for tablet 9fc8d9031ba049389e7a0213cdeddf29 (DEFAULT_TABLE table=test-table [id=7bdb94438617462385e4a3b752826712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:26.083870 26084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fc8d9031ba049389e7a0213cdeddf29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:26.086668 26283 tablet_service.cc:1467] Processing CreateTablet for tablet 9fc8d9031ba049389e7a0213cdeddf29 (DEFAULT_TABLE table=test-table [id=7bdb94438617462385e4a3b752826712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:23:26.087632 26283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fc8d9031ba049389e7a0213cdeddf29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:23:26.091099 26402 tablet_bootstrap.cc:492] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577: Bootstrap starting.
I20260810 07:23:26.094074 26402 tablet_bootstrap.cc:654] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:26.095072 26402 log.cc:824] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:26.095551 26404 tablet_bootstrap.cc:492] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965: Bootstrap starting.
I20260810 07:23:26.097129 26402 tablet_bootstrap.cc:492] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577: No bootstrap required, opened a new log
I20260810 07:23:26.097343 26402 ts_tablet_manager.cc:1397] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:23:26.097858 26404 tablet_bootstrap.cc:654] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965: Neither blocks nor log segments found. Creating new log.
I20260810 07:23:26.098528 26404 log.cc:824] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:26.100440 26404 tablet_bootstrap.cc:492] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965: No bootstrap required, opened a new log
I20260810 07:23:26.100668 26404 ts_tablet_manager.cc:1397] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260810 07:23:26.100888 26402 raft_consensus.cc:348] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:26.101142 26402 raft_consensus.cc:374] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:26.101204 26402 raft_consensus.cc:729] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df52e2524f064df895a02ed44c01f577, State: Initialized, Role: FOLLOWER
I20260810 07:23:26.101662 26402 consensus_queue.cc:260] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:26.102345 26184 heartbeater.cc:503] Master 127.1.125.254:34003 was elected leader, sending a full tablet report...
I20260810 07:23:26.102697 26402 ts_tablet_manager.cc:1428] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:23:26.103376 26404 raft_consensus.cc:348] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:26.103624 26404 raft_consensus.cc:374] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:23:26.103682 26404 raft_consensus.cc:729] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9072591c0a32473cad10cb1d00abb965, State: Initialized, Role: FOLLOWER
I20260810 07:23:26.104097 26404 consensus_queue.cc:260] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:26.104799 26384 heartbeater.cc:503] Master 127.1.125.254:34003 was elected leader, sending a full tablet report...
I20260810 07:23:26.104998 26404 ts_tablet_manager.cc:1428] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
W20260810 07:23:26.123939 25979 tablet.cc:2369] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:23:26.176456 26407 raft_consensus.cc:482] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:26.176715 26407 raft_consensus.cc:504] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:26.178473 26407 leader_election.cc:288] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871), 8aa7968bd68a4b3eb023a62bcad8df51 (127.1.125.193:36935)
I20260810 07:23:26.182829 26186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.178639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:39871 (local address 127.1.125.194:47731)
0810 07:23:26.178800 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:26.178817 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:26.178950 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:26.179262 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:26.179267 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:26.179293 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:26.179434 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:26.179441 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.180593 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.180596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:26.181731 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:26.181742 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.182481 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.182483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:26.182569 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.182592 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:23:26.182656 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:23:26.182713 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":1}
I20260810 07:23:26.182916 26399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.178786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:47731 (local address 127.1.125.195:39871)
0810 07:23:26.178966 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:26.178972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:26.178997 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:23:26.179062 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:26.179066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:26.179128 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:26.179234 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:26.179584 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.180453 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.181861 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.182362 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.182491 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.182549 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:26.182696 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 07:23:26.182775 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 07:23:26.183574 26412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.179064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:36935 (local address 127.1.125.194:39013)
0810 07:23:26.179541 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:26.179555 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:26.179645 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:26.179946 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:26.179948 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:26.179967 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:23:26.180085 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:26.180090 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.181037 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.181041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:26.182113 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:26.182121 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.183159 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.183163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:26.183256 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.183277 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:23:26.183356 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:23:26.183427 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":153,"threads_started":1}
I20260810 07:23:26.183645 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.179134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:39013 (local address 127.1.125.193:36935)
0810 07:23:26.179562 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:26.179567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:26.179609 (+ 42us) server_negotiation.cc:407] Connection header received
0810 07:23:26.179748 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:26.179752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:26.179806 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:26.179912 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:23:26.180209 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.180918 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.182243 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.182894 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.183021 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.183073 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:26.183389 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 07:23:26.183468 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":133,"threads_started":1}
I20260810 07:23:26.183707 26316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fc8d9031ba049389e7a0213cdeddf29" candidate_uuid: "df52e2524f064df895a02ed44c01f577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9072591c0a32473cad10cb1d00abb965" is_pre_election: true
I20260810 07:23:26.184109 26316 raft_consensus.cc:2454] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df52e2524f064df895a02ed44c01f577 in term 0.
I20260810 07:23:26.184219 25910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fc8d9031ba049389e7a0213cdeddf29" candidate_uuid: "df52e2524f064df895a02ed44c01f577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" is_pre_election: true
I20260810 07:23:26.184608 25910 raft_consensus.cc:2454] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df52e2524f064df895a02ed44c01f577 in term 0.
I20260810 07:23:26.184708 26015 leader_election.cc:302] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9072591c0a32473cad10cb1d00abb965, df52e2524f064df895a02ed44c01f577; no voters:
I20260810 07:23:26.185020 26407 raft_consensus.cc:2781] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:23:26.185146 26407 raft_consensus.cc:482] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:23:26.185207 26407 raft_consensus.cc:3037] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:26.186486 26407 raft_consensus.cc:504] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:26.186863 26407 leader_election.cc:288] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [CANDIDATE]: Term 1 election: Requested vote from peers 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871), 8aa7968bd68a4b3eb023a62bcad8df51 (127.1.125.193:36935)
I20260810 07:23:26.187320 26316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fc8d9031ba049389e7a0213cdeddf29" candidate_uuid: "df52e2524f064df895a02ed44c01f577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9072591c0a32473cad10cb1d00abb965"
I20260810 07:23:26.187320 25910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fc8d9031ba049389e7a0213cdeddf29" candidate_uuid: "df52e2524f064df895a02ed44c01f577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aa7968bd68a4b3eb023a62bcad8df51"
I20260810 07:23:26.187459 26316 raft_consensus.cc:3037] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:26.187459 25910 raft_consensus.cc:3037] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:23:26.188838 25910 raft_consensus.cc:2454] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df52e2524f064df895a02ed44c01f577 in term 1.
I20260810 07:23:26.188838 26316 raft_consensus.cc:2454] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df52e2524f064df895a02ed44c01f577 in term 1.
I20260810 07:23:26.189153 26015 leader_election.cc:302] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8aa7968bd68a4b3eb023a62bcad8df51, df52e2524f064df895a02ed44c01f577; no voters:
I20260810 07:23:26.189353 26407 raft_consensus.cc:2781] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:23:26.189618 26407 raft_consensus.cc:686] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [term 1 LEADER]: Becoming Leader. State: Replica: df52e2524f064df895a02ed44c01f577, State: Running, Role: LEADER
I20260810 07:23:26.189960 26407 consensus_queue.cc:237] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:26.193470 25684 catalog_manager.cc:5495] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 reported cstate change: term changed from 0 to 1, leader changed from <none> to df52e2524f064df895a02ed44c01f577 (127.1.125.194). New cstate: current_term: 1 leader_uuid: "df52e2524f064df895a02ed44c01f577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } health_report { overall_health: UNKNOWN } } }
I20260810 07:23:26.204607 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.200903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.1.125.194:44919)
0810 07:23:26.201030 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:26.201034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:26.201122 (+ 88us) server_negotiation.cc:407] Connection header received
0810 07:23:26.201319 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:26.201322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:26.201378 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:26.201500 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:23:26.201747 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.202254 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.203116 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:26.203342 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:26.203452 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:26.203773 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:23:26.204194 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:23:26.204334 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:26.204375 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 07:23:26.204432 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:23:26.210803 25910 raft_consensus.cc:1261] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Refusing update from remote peer df52e2524f064df895a02ed44c01f577: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:26.210897 26316 raft_consensus.cc:1261] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965 [term 1 FOLLOWER]: Refusing update from remote peer df52e2524f064df895a02ed44c01f577: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:23:26.211570 26416 consensus_queue.cc:1035] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:26.211738 26407 consensus_queue.cc:1035] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:23:26.217397 26421 mvcc.cc:205] Tried to move back new op lower bound from 7316875699031322624 to 7316875698956873728. Current Snapshot: MvccSnapshot[applied={T|T < 7316875699031322624}]
I20260810 07:23:26.219324 26423 mvcc.cc:205] Tried to move back new op lower bound from 7316875699031322624 to 7316875698956873728. Current Snapshot: MvccSnapshot[applied={T|T < 7316875699031322624}]
I20260810 07:23:26.228381 25685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:26.228612 25685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:23:26.231175 25685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Sending DeleteTablet for 3 replicas of tablet 9fc8d9031ba049389e7a0213cdeddf29
I20260810 07:23:26.231886 26283 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fc8d9031ba049389e7a0213cdeddf29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:26 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:41998
I20260810 07:23:26.231994 26084 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fc8d9031ba049389e7a0213cdeddf29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:26 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:54934
I20260810 07:23:26.232132 25878 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fc8d9031ba049389e7a0213cdeddf29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:23:26 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:40162
I20260810 07:23:26.232513 26441 tablet_replica.cc:330] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577: stopping tablet replica
I20260810 07:23:26.232548 26438 tablet_replica.cc:330] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965: stopping tablet replica
I20260810 07:23:26.232844 26441 raft_consensus.cc:2229] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:23:26.232873 26438 raft_consensus.cc:2229] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:23:26.232892 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 25756
I20260810 07:23:26.235391 26441 raft_consensus.cc:2258] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:26.241760 26441 ts_tablet_manager.cc:1905] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:26.245229 26441 ts_tablet_manager.cc:1918] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:26.245332 26441 log.cc:1196] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/9fc8d9031ba049389e7a0213cdeddf29
I20260810 07:23:26.245647 26441 ts_tablet_manager.cc:1939] T 9fc8d9031ba049389e7a0213cdeddf29 P df52e2524f064df895a02ed44c01f577: Deleting consensus metadata
I20260810 07:23:26.246876 25668 catalog_manager.cc:4841] TS df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919): tablet 9fc8d9031ba049389e7a0213cdeddf29 (table test-table [id=7bdb94438617462385e4a3b752826712]) successfully deleted
I20260810 07:23:26.249706 26438 raft_consensus.cc:2258] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:23:26.251636 26438 ts_tablet_manager.cc:1905] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:23:26.254511 26438 ts_tablet_manager.cc:1918] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:23:26.254619 26438 log.cc:1196] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/9fc8d9031ba049389e7a0213cdeddf29
I20260810 07:23:26.254979 26438 ts_tablet_manager.cc:1939] T 9fc8d9031ba049389e7a0213cdeddf29 P 9072591c0a32473cad10cb1d00abb965: Deleting consensus metadata
I20260810 07:23:26.256376 25666 catalog_manager.cc:4841] TS 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871): tablet 9fc8d9031ba049389e7a0213cdeddf29 (table test-table [id=7bdb94438617462385e4a3b752826712]) successfully deleted
W20260810 07:23:26.257126 25666 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:36935 (error 108)
I20260810 07:23:26.257812 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 25985
I20260810 07:23:26.262629 26397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:26.257986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:36935 (local address 127.0.0.1:40174)
0810 07:23:26.262430 (+ 4444us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:26.262523 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:36935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4327}
W20260810 07:23:26.274534 25666 catalog_manager.cc:4584] TS 8aa7968bd68a4b3eb023a62bcad8df51 (127.1.125.193:36935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9fc8d9031ba049389e7a0213cdeddf29: Network error: Client connection negotiation failed: client connection to 127.1.125.193:36935: connect: Connection refused (error 111)
I20260810 07:23:26.281499 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 26187
I20260810 07:23:26.303822 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 25634
I20260810 07:23:46.317830 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:34003
--webserver_interface=127.1.125.254
--webserver_port=42147
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:40763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:34003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:46.436887 26445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:46.437153 26445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:46.437237 26445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:46.441390 26445 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 07:23:46.441449 26445 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 07:23:46.441468 26445 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 07:23:46.441481 26445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:23:46.441540 26445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:46.441573 26445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:46.441606 26445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:23:46.441640 26445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:23:46.446851 26445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:40763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:34003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:34003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=42147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:23:46.447902 26445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:46.448958 26445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:46.457849 26455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:46.457943 26458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:46.457952 26454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:46.458204 26445 server_base.cc:1034] running on GCE node
I20260810 07:23:46.458752 26445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:46.459918 26445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:46.461089 26445 hybrid_clock.cc:648] HybridClock initialized: now 1786346626461057 us; error 49 us; skew 500 ppm
Aug 10 07:23:46 dist-test-slave-44d2 krb5kdc[24820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346626, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:46.464175 26445 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:23:46.465265 26445 webserver.cc:460] Webserver started at http://127.1.125.254:42147/ using document root <none> and password file <none>
I20260810 07:23:46.465874 26445 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:46.465967 26445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:46.470145 26445 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260810 07:23:46.472472 26468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:46.473906 26445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:23:46.474100 26445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "4978926106d341a6a9962256304bc518"
format_stamp: "Formatted at 2026-08-10 07:23:03 on dist-test-slave-44d2"
server_key: "5a594dc1fe586cbbdf2fff07c816a52b"
server_key_iv: "42513ba2ffab905ed844d0351985cf73"
server_key_version: "encryptionkey@0"
I20260810 07:23:46.474735 26445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:46.501914 26445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:46.505262 26445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:46.505509 26445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:46.513476 26445 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:34003
I20260810 07:23:46.513787 26546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:34003 every 8 connection(s)
I20260810 07:23:46.514705 26445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:23:46.516291 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 26445
I20260810 07:23:46.516878 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:36935
--local_ip_for_outbound_sockets=127.1.125.193
--tserver_master_addrs=127.1.125.254:34003
--webserver_port=44927
--webserver_interface=127.1.125.193
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:40763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:23:46.520200 26547 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:23:46.521955 26547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Bootstrap starting.
I20260810 07:23:46.543062 26547 log.cc:824] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:46.555099 26547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:23:46.555567 26547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Bootstrap complete.
I20260810 07:23:46.560127 26547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } }
I20260810 07:23:46.560694 26547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4978926106d341a6a9962256304bc518, State: Initialized, Role: FOLLOWER
I20260810 07:23:46.561255 26547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } }
I20260810 07:23:46.561399 26547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:23:46.561508 26547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:23:46.561635 26547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:23:46.563805 26547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } }
I20260810 07:23:46.564204 26547 leader_election.cc:302] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4978926106d341a6a9962256304bc518; no voters:
I20260810 07:23:46.564643 26547 leader_election.cc:288] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 07:23:46.564743 26553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 07:23:46.564977 26553 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [term 3 LEADER]: Becoming Leader. State: Replica: 4978926106d341a6a9962256304bc518, State: Running, Role: LEADER
I20260810 07:23:46.565299 26553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } }
I20260810 07:23:46.566028 26547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:23:46.566828 26555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4978926106d341a6a9962256304bc518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } } }
I20260810 07:23:46.566993 26555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:46.567595 26556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4978926106d341a6a9962256304bc518. Latest consensus state: current_term: 3 leader_uuid: "4978926106d341a6a9962256304bc518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978926106d341a6a9962256304bc518" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 34003 } } }
I20260810 07:23:46.567759 26556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518 [sys.catalog]: This master's current role is: LEADER
I20260810 07:23:46.568073 26567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:23:46.573426 26567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:23:46.574054 26567 catalog_manager.cc:1259] Loaded cluster ID: 179fd7238e484fe7b99425dc87612281
I20260810 07:23:46.574117 26567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:23:46.576614 26567 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:23:46.577322 26567 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Loaded TSK: 0
I20260810 07:23:46.577423 26567 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4978926106d341a6a9962256304bc518: Loaded TSK: 1
F20260810 07:23:46.578197 26567 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f052dd869ad google::LogMessage::Fail() at ??:0
@ 0x7f052dd8988a google::LogMessage::SendToLog() at ??:0
@ 0x7f052dd864de google::LogMessage::Flush() at ??:0
@ 0x7f052dd87a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0530a4a18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0530a42f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f05386eb7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f05386c599c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f05386c6178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f0538707df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f053872682b std::function<>::operator()() at ??:0
@ 0x7f05386c5b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f05386c6f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f05386c3579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f0538707640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f052ffd83d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f052ffd8c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f052ffda3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f052ffc9512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f053160d6db start_thread at ??:0
@ 0x7f052c88461f clone at ??:0
W20260810 07:23:46.680543 26550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:46.680932 26550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:46.681010 26550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:46.687491 26550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:46.687578 26550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:46.687685 26550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:23:46.696128 26550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:40763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:36935
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=44927
--tserver_master_addrs=127.1.125.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:23:46.697640 26550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:46.699055 26550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:46.710975 26624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:46.710995 26622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:46.710992 26620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:46.711427 26550 server_base.cc:1034] running on GCE node
I20260810 07:23:46.712033 26550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:46.712757 26550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:46.713939 26550 hybrid_clock.cc:648] HybridClock initialized: now 1786346626713917 us; error 43 us; skew 500 ppm
Aug 10 07:23:46 dist-test-slave-44d2 krb5kdc[24820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346626, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:46.717234 26550 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:23:46.718212 26550 webserver.cc:460] Webserver started at http://127.1.125.193:44927/ using document root <none> and password file <none>
I20260810 07:23:46.718791 26550 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:46.718854 26550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:46.722971 26550 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:23:46.725499 26637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:46.726876 26550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:23:46.727128 26550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "8aa7968bd68a4b3eb023a62bcad8df51"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "13fff1810b5ef56d60e97c9428535813"
server_key_iv: "d0d7ac9b704b4026d73ef4bee48e6754"
server_key_version: "encryptionkey@0"
I20260810 07:23:46.727705 26550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:46.763463 26550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:46.767153 26550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:46.767386 26550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:46.768079 26550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:46.769380 26648 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:23:46.775319 26550 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:23:46.775408 26550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260810 07:23:46.775488 26550 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:23:46.777346 26550 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:23:46.777431 26550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260810 07:23:46.777905 26648 tablet_bootstrap.cc:492] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51: Bootstrap starting.
I20260810 07:23:46.790818 26550 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:36935
I20260810 07:23:46.791944 26550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:23:46.793351 26814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:36935 every 8 connection(s)
I20260810 07:23:46.798337 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 26550
I20260810 07:23:46.798822 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:44919
--local_ip_for_outbound_sockets=127.1.125.194
--tserver_master_addrs=127.1.125.254:34003
--webserver_port=43025
--webserver_interface=127.1.125.194
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:40763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:23:46.802006 26648 log.cc:824] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51: Log is configured to *not* fsync() on all Append() calls
I20260810 07:23:46.809473 26648 tablet_bootstrap.cc:492] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:23:46.809880 26648 tablet_bootstrap.cc:492] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51: Bootstrap complete.
I20260810 07:23:46.811170 26648 ts_tablet_manager.cc:1397] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51: Time spent bootstrapping tablet: real 0.034s user 0.022s sys 0.009s
I20260810 07:23:46.813664 26648 raft_consensus.cc:348] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:46.814105 26648 raft_consensus.cc:729] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aa7968bd68a4b3eb023a62bcad8df51, State: Initialized, Role: FOLLOWER
I20260810 07:23:46.814597 26648 consensus_queue.cc:260] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:46.815598 26648 ts_tablet_manager.cc:1428] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
W20260810 07:23:46.858901 26822 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:23:46.794417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:56597)
0810 07:23:46.794983 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:46.795042 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:46.796000 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:46.858062 (+ 62062us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":179,"threads_started":1}
W20260810 07:23:46.859606 26815 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:34003 (0 consecutive failures): Network error: Failed to ping master at 127.1.125.254:34003: Client connection negotiation failed: client connection to 127.1.125.254:34003: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:23:46.860447 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:46.860170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:51715)
0810 07:23:46.860311 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:46.860390 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 07:23:46.861232 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:46.861012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:43229)
0810 07:23:46.861126 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:46.861182 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 07:23:46.861444 26815 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:23:46.936299 26824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:46.936647 26824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:46.936744 26824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:46.940919 26824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:46.941010 26824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:46.941108 26824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:23:46.946368 26824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:40763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:44919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=43025
--tserver_master_addrs=127.1.125.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:23:46.947471 26824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:46.948567 26824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:46.958909 26837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:46.958868 26835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:46.959007 26834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:46.959251 26824 server_base.cc:1034] running on GCE node
I20260810 07:23:46.959667 26824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:46.960361 26824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:46.961580 26824 hybrid_clock.cc:648] HybridClock initialized: now 1786346626961565 us; error 27 us; skew 500 ppm
Aug 10 07:23:46 dist-test-slave-44d2 krb5kdc[24820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346626, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:46.970163 26824 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:23:46.971105 26824 webserver.cc:460] Webserver started at http://127.1.125.194:43025/ using document root <none> and password file <none>
I20260810 07:23:46.971644 26824 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:46.971704 26824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:46.975703 26824 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:23:46.977862 26846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:46.979131 26824 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:23:46.979265 26824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "df52e2524f064df895a02ed44c01f577"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "c76f831350aed5ad7be217c8e7563dd1"
server_key_iv: "149f14e0e8452f63afb462f52de2ccbe"
server_key_version: "encryptionkey@0"
I20260810 07:23:46.979813 26824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:47.004374 26824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:47.008247 26824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:47.008535 26824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:47.009266 26824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:47.010211 26824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:47.010267 26824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:47.010314 26824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:47.010385 26824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:47.021626 26824 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:44919
I20260810 07:23:47.021761 27024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:44919 every 8 connection(s)
I20260810 07:23:47.022773 26824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:23:47.026989 27032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:47.025317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:58477)
0810 07:23:47.026126 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:47.026241 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":137,"threads_started":1}
W20260810 07:23:47.027482 27028 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:34003 (0 consecutive failures): Network error: Failed to ping master at 127.1.125.254:34003: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
I20260810 07:23:47.028136 27032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:47.027870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:41539)
0810 07:23:47.028028 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:47.028079 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 07:23:47.028944 27032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:47.028728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:41481)
0810 07:23:47.028841 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:47.028893 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 07:23:47.029172 27028 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:23:47.031803 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 26824
I20260810 07:23:47.032375 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:39871
--local_ip_for_outbound_sockets=127.1.125.195
--tserver_master_addrs=127.1.125.254:34003
--webserver_port=45763
--webserver_interface=127.1.125.195
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:40763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:23:47.147115 27034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:23:47.147418 27034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:23:47.147495 27034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:23:47.151472 27034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:23:47.151572 27034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:23:47.151670 27034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:23:47.156812 27034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:40763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:39871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=45763
--tserver_master_addrs=127.1.125.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:23:47.157833 27034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:23:47.158861 27034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:23:47.168577 27039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:47.168578 27042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:23:47.168577 27040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:23:47.168984 27034 server_base.cc:1034] running on GCE node
I20260810 07:23:47.169384 27034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:23:47.169986 27034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:23:47.171173 27034 hybrid_clock.cc:648] HybridClock initialized: now 1786346627171158 us; error 27 us; skew 500 ppm
Aug 10 07:23:47 dist-test-slave-44d2 krb5kdc[24820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346627, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:23:47.174070 27034 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:23:47.174937 27034 webserver.cc:460] Webserver started at http://127.1.125.195:45763/ using document root <none> and password file <none>
I20260810 07:23:47.175513 27034 fs_manager.cc:359] Metadata directory not provided
I20260810 07:23:47.175580 27034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:23:47.179678 27034 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:23:47.182094 27054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:47.183621 27034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 07:23:47.183766 27034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "9072591c0a32473cad10cb1d00abb965"
format_stamp: "Formatted at 2026-08-10 07:23:04 on dist-test-slave-44d2"
server_key: "23db91b6c22a4848a75b4008b7298a4a"
server_key_iv: "8bc1aa380678549c2ece4ae5ee070f1d"
server_key_version: "encryptionkey@0"
I20260810 07:23:47.184473 27034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:23:47.206147 27034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:23:47.209378 27034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:23:47.209601 27034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:23:47.210314 27034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:23:47.211253 27034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:23:47.211331 27034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:47.211409 27034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:23:47.211462 27034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:23:47.221323 27034 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:39871
I20260810 07:23:47.221360 27232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:39871 every 8 connection(s)
I20260810 07:23:47.222337 27034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:23:47.226311 27235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:47.224688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:44467)
0810 07:23:47.225450 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:47.225560 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":138,"threads_started":1}
W20260810 07:23:47.226858 27233 heartbeater.cc:644] Failed to heartbeat to 127.1.125.254:34003 (0 consecutive failures): Network error: Failed to ping master at 127.1.125.254:34003: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
I20260810 07:23:47.227490 27235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:47.227234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:41525)
0810 07:23:47.227346 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:47.227396 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 07:23:47.228184 27235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:47.227958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:48547)
0810 07:23:47.228076 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:47.228134 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 07:23:47.228367 27233 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:23:47.230392 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 27034
I20260810 07:23:47.862568 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:47.862067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:40045)
0810 07:23:47.862372 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:47.862435 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":80,"threads_started":1}
I20260810 07:23:48.030236 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:48.029795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:39401)
0810 07:23:48.030049 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:48.030125 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":75,"threads_started":1}
I20260810 07:23:48.229336 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:48.228907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:35587)
0810 07:23:48.229174 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:48.229232 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":72,"threads_started":1}
I20260810 07:23:48.308879 27245 raft_consensus.cc:482] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:48.309128 27245 raft_consensus.cc:504] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:48.310371 27245 leader_election.cc:288] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871), df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919)
Aug 10 07:23:48 dist-test-slave-44d2 krb5kdc[24820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346626, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.194@KRBTEST.COM
Aug 10 07:23:48 dist-test-slave-44d2 krb5kdc[24820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346626, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.195@KRBTEST.COM
I20260810 07:23:48.325376 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:48.310573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:39871 (local address 127.1.125.193:48125)
0810 07:23:48.310951 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:48.310969 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:48.311096 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:48.312925 (+ 1829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:48.312931 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:48.313499 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:48.313997 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:48.314011 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:48.315320 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:48.315324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:48.316237 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:48.316248 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:48.317070 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:48.317074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:48.317283 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:48.318025 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:48.318038 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:48.320004 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:48.322374 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:48.322377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:48.322379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:48.322668 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:48.323103 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:48.323107 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:48.323109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:48.323312 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:48.324141 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:48.324148 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:48.324526 (+ 378us) client_negotiation.cc:769] Sending connection context
0810 07:23:48.324752 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 07:23:48.325159 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":242,"spinlock_wait_cycles":56064}
I20260810 07:23:48.325392 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:48.310756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:48125 (local address 127.1.125.195:39871)
0810 07:23:48.311080 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:48.311127 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:48.311177 (+ 50us) server_negotiation.cc:407] Connection header received
0810 07:23:48.312097 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:48.312105 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:48.312459 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:48.312831 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:48.314189 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:48.315157 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:48.316389 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:48.316928 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:48.317118 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:48.320207 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:48.320212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:48.320223 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:48.320251 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:48.322260 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:48.322786 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:48.322793 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:48.322799 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:48.322969 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:48.323433 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:48.323436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:48.323438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:48.323968 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:48.324106 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:48.324897 (+ 791us) server_negotiation.cc:299] Negotiation successful
0810 07:23:48.325201 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":130,"threads_started":1}
I20260810 07:23:48.325393 27250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:48.310827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:45023 (local address 127.1.125.194:44919)
0810 07:23:48.311381 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0810 07:23:48.311406 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 07:23:48.311447 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:23:48.312211 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:23:48.312218 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:23:48.312609 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:23:48.312913 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:23:48.314192 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:48.315161 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:48.316367 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:23:48.316931 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:23:48.317125 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:48.319914 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:23:48.319923 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:23:48.319940 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:23:48.319978 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:23:48.322028 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:48.322786 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:48.322793 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:48.322799 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:48.322969 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:23:48.323433 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:23:48.323436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:23:48.323438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:23:48.323968 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:23:48.324114 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 07:23:48.324897 (+ 783us) server_negotiation.cc:299] Negotiation successful
0810 07:23:48.325201 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":118,"threads_started":1}
I20260810 07:23:48.325374 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:48.310639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:44919 (local address 127.1.125.193:45023)
0810 07:23:48.311145 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:48.311160 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:23:48.311258 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:23:48.312926 (+ 1668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:23:48.312932 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:23:48.313498 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:23:48.313999 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:48.314011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:48.315324 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:48.315328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:48.316231 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:23:48.316241 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:23:48.317066 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:23:48.317069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:23:48.317294 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:23:48.317968 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:23:48.318015 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:23:48.319641 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:23:48.322171 (+ 2530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:48.322177 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:48.322191 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:48.322637 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:48.323110 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:23:48.323112 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:23:48.323115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:23:48.323312 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:23:48.324141 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:23:48.324148 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:23:48.324526 (+ 378us) client_negotiation.cc:769] Sending connection context
0810 07:23:48.324752 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 07:23:48.325213 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":294,"spinlock_wait_cycles":16768,"thread_start_us":64,"threads_started":1}
W20260810 07:23:48.328089 26642 leader_election.cc:341] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919): Not found: Tablet not found: 9fc8d9031ba049389e7a0213cdeddf29
W20260810 07:23:48.328293 26640 leader_election.cc:341] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871): Not found: Tablet not found: 9fc8d9031ba049389e7a0213cdeddf29
I20260810 07:23:48.328404 26640 leader_election.cc:302] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8aa7968bd68a4b3eb023a62bcad8df51; no voters: 9072591c0a32473cad10cb1d00abb965, df52e2524f064df895a02ed44c01f577
I20260810 07:23:48.328694 27245 raft_consensus.cc:2726] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:23:48.863945 27252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:48.863467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:40851)
0810 07:23:48.863756 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:48.863816 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":83,"threads_started":1}
I20260810 07:23:49.031399 27253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:49.031016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:46147)
0810 07:23:49.031255 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:49.031308 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":73,"threads_started":1}
I20260810 07:23:49.230557 27255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:49.230090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:38367)
0810 07:23:49.230397 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:49.230464 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":102,"threads_started":1}
I20260810 07:23:49.834748 27256 raft_consensus.cc:482] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:49.834949 27256 raft_consensus.cc:504] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:49.835505 27256 leader_election.cc:288] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871), df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919)
W20260810 07:23:49.836606 26642 leader_election.cc:341] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919): Not found: Tablet not found: 9fc8d9031ba049389e7a0213cdeddf29
W20260810 07:23:49.836699 26640 leader_election.cc:341] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871): Not found: Tablet not found: 9fc8d9031ba049389e7a0213cdeddf29
I20260810 07:23:49.836767 26640 leader_election.cc:302] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8aa7968bd68a4b3eb023a62bcad8df51; no voters: 9072591c0a32473cad10cb1d00abb965, df52e2524f064df895a02ed44c01f577
I20260810 07:23:49.836942 27256 raft_consensus.cc:2726] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:23:49.865468 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:49.864995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:52275)
0810 07:23:49.865305 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:49.865365 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":94,"threads_started":1}
I20260810 07:23:50.032748 27260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:50.032186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:58679)
0810 07:23:50.032490 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:50.032626 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":112,"threads_started":1}
I20260810 07:23:50.231775 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:50.231332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:42003)
0810 07:23:50.231584 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:50.231639 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
I20260810 07:23:50.866850 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:50.866359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:52619)
0810 07:23:50.866673 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:50.866735 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":101,"threads_started":1}
I20260810 07:23:51.034052 27263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:51.033578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:34099)
0810 07:23:51.033846 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:51.033900 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":97,"threads_started":1}
I20260810 07:23:51.233074 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:51.232650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:39869)
0810 07:23:51.232920 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:51.232976 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":80,"threads_started":1}
I20260810 07:23:51.868722 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:51.868080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:54599)
0810 07:23:51.868469 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:51.868563 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":97,"threads_started":1}
I20260810 07:23:51.884940 27267 raft_consensus.cc:482] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:51.885092 27267 raft_consensus.cc:504] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:51.885674 27267 leader_election.cc:288] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871), df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919)
W20260810 07:23:51.886588 26642 leader_election.cc:341] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919): Not found: Tablet not found: 9fc8d9031ba049389e7a0213cdeddf29
W20260810 07:23:51.886687 26640 leader_election.cc:341] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871): Not found: Tablet not found: 9fc8d9031ba049389e7a0213cdeddf29
I20260810 07:23:51.886744 26642 leader_election.cc:302] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8aa7968bd68a4b3eb023a62bcad8df51; no voters: 9072591c0a32473cad10cb1d00abb965, df52e2524f064df895a02ed44c01f577
I20260810 07:23:51.886925 27267 raft_consensus.cc:2726] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:23:52.035218 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:52.034801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:45735)
0810 07:23:52.035053 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:52.035126 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
I20260810 07:23:52.234393 27269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:52.233942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:49865)
0810 07:23:52.234241 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:52.234295 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":120,"threads_started":1}
I20260810 07:23:52.870020 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:52.869525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:55507)
0810 07:23:52.869843 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:52.869906 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":109,"threads_started":1}
I20260810 07:23:53.036338 27273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:53.035914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:42625)
0810 07:23:53.036196 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:53.036248 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":113,"threads_started":1}
I20260810 07:23:53.235615 27276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:53.235162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:36539)
0810 07:23:53.235445 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:53.235514 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":99,"threads_started":1}
I20260810 07:23:53.871272 27277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:53.870817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:54075)
0810 07:23:53.871109 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:53.871171 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":82,"threads_started":1}
I20260810 07:23:54.037626 27280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:54.037089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:55869)
0810 07:23:54.037447 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:54.037524 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":110,"threads_started":1}
I20260810 07:23:54.236979 27284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:54.236443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:33793)
0810 07:23:54.236817 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:54.236878 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":76,"threads_started":1}
I20260810 07:23:54.872788 27285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:54.872204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:50693)
0810 07:23:54.872589 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:54.872669 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":92,"threads_started":1}
I20260810 07:23:55.038980 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:55.038510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:34127)
0810 07:23:55.038799 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:55.038864 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":85,"threads_started":1}
I20260810 07:23:55.238332 27288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:55.237848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:36419)
0810 07:23:55.238171 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:55.238229 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":103,"threads_started":1}
I20260810 07:23:55.874147 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:55.873672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:47561)
0810 07:23:55.873967 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:55.874047 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":110,"threads_started":1}
I20260810 07:23:56.040313 27290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:56.039814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:39591)
0810 07:23:56.040129 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:56.040200 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":81,"threads_started":1}
I20260810 07:23:56.223452 27291 raft_consensus.cc:482] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:23:56.223629 27291 raft_consensus.cc:504] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:23:56.224117 27291 leader_election.cc:288] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871), df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919)
W20260810 07:23:56.224984 26640 leader_election.cc:341] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871): Not found: Tablet not found: 9fc8d9031ba049389e7a0213cdeddf29
W20260810 07:23:56.225109 26642 leader_election.cc:341] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919): Not found: Tablet not found: 9fc8d9031ba049389e7a0213cdeddf29
I20260810 07:23:56.225237 26642 leader_election.cc:302] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8aa7968bd68a4b3eb023a62bcad8df51; no voters: 9072591c0a32473cad10cb1d00abb965, df52e2524f064df895a02ed44c01f577
I20260810 07:23:56.225437 27291 raft_consensus.cc:2726] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:23:56.239665 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:56.239111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:41549)
0810 07:23:56.239487 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:56.239548 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":94,"threads_started":1}
I20260810 07:23:56.875519 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:56.875082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:54439)
0810 07:23:56.875357 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:56.875414 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":81,"threads_started":1}
I20260810 07:23:57.041565 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:57.041144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:38575)
0810 07:23:57.041411 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:57.041465 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":78,"threads_started":1}
I20260810 07:23:57.240995 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:57.240585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:54771)
0810 07:23:57.240840 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:57.240899 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
I20260810 07:23:57.876765 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:57.876312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:41119)
0810 07:23:57.876608 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:57.876662 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":83,"threads_started":1}
I20260810 07:23:58.042737 27301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:58.042305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:51949)
0810 07:23:58.042582 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:58.042636 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":77,"threads_started":1}
I20260810 07:23:58.242388 27302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:58.241859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:38723)
0810 07:23:58.242187 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:58.242258 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":102,"threads_started":1}
I20260810 07:23:58.877945 27303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:58.877523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:47135)
0810 07:23:58.877786 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:58.877844 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":93,"threads_started":1}
I20260810 07:23:59.044034 27304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:59.043572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:42615)
0810 07:23:59.043881 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:59.043935 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":109,"threads_started":1}
I20260810 07:23:59.243602 27305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:59.243176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:36937)
0810 07:23:59.243430 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:59.243489 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260810 07:23:59.879081 27307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:23:59.878683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:38519)
0810 07:23:59.878924 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 07:23:59.878974 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":81,"threads_started":1}
I20260810 07:24:00.045284 27308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:00.044831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:33329)
0810 07:24:00.045121 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:00.045180 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":96,"threads_started":1}
I20260810 07:24:00.245059 27315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:00.244606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:51115)
0810 07:24:00.244895 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:00.244958 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":100,"threads_started":1}
I20260810 07:24:00.880308 27318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:00.879851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:38709)
0810 07:24:00.880145 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:00.880204 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":79,"threads_started":1}
I20260810 07:24:01.046522 27320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:01.046045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:50869)
0810 07:24:01.046371 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:01.046423 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":129,"threads_started":1}
I20260810 07:24:01.189910 27322 raft_consensus.cc:482] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:24:01.190085 27322 raft_consensus.cc:504] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9072591c0a32473cad10cb1d00abb965" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 39871 } } peers { permanent_uuid: "df52e2524f064df895a02ed44c01f577" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44919 } } peers { permanent_uuid: "8aa7968bd68a4b3eb023a62bcad8df51" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 36935 } }
I20260810 07:24:01.190559 27322 leader_election.cc:288] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871), df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919)
W20260810 07:24:01.191419 26642 leader_election.cc:341] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df52e2524f064df895a02ed44c01f577 (127.1.125.194:44919): Not found: Tablet not found: 9fc8d9031ba049389e7a0213cdeddf29
W20260810 07:24:01.191525 26640 leader_election.cc:341] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9072591c0a32473cad10cb1d00abb965 (127.1.125.195:39871): Not found: Tablet not found: 9fc8d9031ba049389e7a0213cdeddf29
I20260810 07:24:01.191593 26640 leader_election.cc:302] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8aa7968bd68a4b3eb023a62bcad8df51; no voters: 9072591c0a32473cad10cb1d00abb965, df52e2524f064df895a02ed44c01f577
I20260810 07:24:01.191756 27322 raft_consensus.cc:2726] T 9fc8d9031ba049389e7a0213cdeddf29 P 8aa7968bd68a4b3eb023a62bcad8df51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:24:01.246423 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:01.245970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:47055)
0810 07:24:01.246268 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:01.246326 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":89,"threads_started":1}
I20260810 07:24:01.881695 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:01.881266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.193:52081)
0810 07:24:01.881519 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:01.881574 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":76,"threads_started":1}
I20260810 07:24:02.047844 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:02.047445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.194:51503)
0810 07:24:02.047693 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:02.047747 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
I20260810 07:24:02.232188 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 26550
I20260810 07:24:02.247993 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:02.247544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:34003 (local address 127.1.125.195:50727)
0810 07:24:02.247832 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:02.247892 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.254:34003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":96,"threads_started":1}
I20260810 07:24:02.248632 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 26824
I20260810 07:24:02.268821 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 27034
2026-08-10T07:24:02Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59909 ms)
[----------] 30 tests from SecurityITest (255816 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:02 dist-test-slave-44d2 krb5kdc[27332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:02 dist-test-slave-44d2 krb5kdc[27332](info): set up 2 sockets
Aug 10 07:24:02 dist-test-slave-44d2 krb5kdc[27332](info): commencing operation
krb5kdc: starting...
W20260810 07:24:03.621639 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:03 dist-test-slave-44d2 krb5kdc[27332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:24:03.761122 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:40033
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:40033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:03.875042 27351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:03.875361 27351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:03.875438 27351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:03.879537 27351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:03.879624 27351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:03.879663 27351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:03.879721 27351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:03.879760 27351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:03.885368 27351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:40033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:40033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:03.886574 27351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:03.887624 27351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:03.896821 27364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:03.896807 27358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:03.896855 27351 server_base.cc:1034] running on GCE node
W20260810 07:24:03.896807 27362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:03.897545 27351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:03.898681 27351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:03.899876 27351 hybrid_clock.cc:648] HybridClock initialized: now 1786346643899857 us; error 31 us; skew 500 ppm
Aug 10 07:24:03 dist-test-slave-44d2 krb5kdc[27332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346643, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:03.902642 27351 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:24:03.903596 27351 webserver.cc:460] Webserver started at http://127.1.125.254:38259/ using document root <none> and password file <none>
I20260810 07:24:03.904114 27351 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:03.904174 27351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:03.904340 27351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:03.906361 27351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "6dfa57c51b4346b6bb56067a83fea9d1"
format_stamp: "Formatted at 2026-08-10 07:24:03 on dist-test-slave-44d2"
server_key: "53de011ecd3a404d8e6ccc11fc1c10bc"
server_key_iv: "81ba5432a6811290fb6ac7318f323d07"
server_key_version: "encryptionkey@0"
I20260810 07:24:03.906881 27351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "6dfa57c51b4346b6bb56067a83fea9d1"
format_stamp: "Formatted at 2026-08-10 07:24:03 on dist-test-slave-44d2"
server_key: "53de011ecd3a404d8e6ccc11fc1c10bc"
server_key_iv: "81ba5432a6811290fb6ac7318f323d07"
server_key_version: "encryptionkey@0"
I20260810 07:24:03.910570 27351 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:24:03.912938 27376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:03.914201 27351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:24:03.914366 27351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "6dfa57c51b4346b6bb56067a83fea9d1"
format_stamp: "Formatted at 2026-08-10 07:24:03 on dist-test-slave-44d2"
server_key: "53de011ecd3a404d8e6ccc11fc1c10bc"
server_key_iv: "81ba5432a6811290fb6ac7318f323d07"
server_key_version: "encryptionkey@0"
I20260810 07:24:03.914476 27351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:03.939359 27351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:03.942375 27351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:03.942592 27351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:03.950084 27351 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:40033
I20260810 07:24:03.950110 27464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:40033 every 8 connection(s)
I20260810 07:24:03.951045 27351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:03.953923 27465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:03.958076 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 27351
I20260810 07:24:03.958247 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:03.958633 1527 external_mini_cluster.cc:1428] Setting key 79f42b34e7106a67a446e63bd6363a96
I20260810 07:24:03.961187 27465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1: Bootstrap starting.
I20260810 07:24:03.963759 27465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:03.964694 27465 log.cc:824] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:03.967293 27465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1: No bootstrap required, opened a new log
Aug 10 07:24:03 dist-test-slave-44d2 krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:24:03.971653 27465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfa57c51b4346b6bb56067a83fea9d1" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40033 } }
I20260810 07:24:03.971904 27465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:03.972003 27465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dfa57c51b4346b6bb56067a83fea9d1, State: Initialized, Role: FOLLOWER
I20260810 07:24:03.972561 27465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfa57c51b4346b6bb56067a83fea9d1" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40033 } }
I20260810 07:24:03.972776 27465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:03.972874 27465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:03.973021 27465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:03.974059 27465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfa57c51b4346b6bb56067a83fea9d1" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40033 } }
I20260810 07:24:03.974510 27465 leader_election.cc:302] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dfa57c51b4346b6bb56067a83fea9d1; no voters:
I20260810 07:24:03.974860 27465 leader_election.cc:288] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:03.974967 27472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:03.975201 27472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [term 1 LEADER]: Becoming Leader. State: Replica: 6dfa57c51b4346b6bb56067a83fea9d1, State: Running, Role: LEADER
I20260810 07:24:03.975382 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:03.960291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36598 (local address 127.1.125.254:40033)
0810 07:24:03.960800 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:03.960816 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:03.960853 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:24:03.961559 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:03.961578 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:03.961916 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:03.962269 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:03.963120 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:03.964201 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:03.965334 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:03.966178 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:03.966393 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:03.969513 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:03.969519 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:03.969533 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:03.969562 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:03.971913 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:03.972533 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:03.972539 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:03.972545 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:03.972714 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:03.973039 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:03.973043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:03.973044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:03.973517 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:03.973740 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:03.974026 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 07:24:03.974347 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":180,"threads_started":1}
I20260810 07:24:03.975592 27472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfa57c51b4346b6bb56067a83fea9d1" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40033 } }
I20260810 07:24:03.976409 27465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:03.977624 27473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dfa57c51b4346b6bb56067a83fea9d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfa57c51b4346b6bb56067a83fea9d1" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40033 } } }
I20260810 07:24:03.977866 27473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:03.976948 27474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dfa57c51b4346b6bb56067a83fea9d1. Latest consensus state: current_term: 1 leader_uuid: "6dfa57c51b4346b6bb56067a83fea9d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfa57c51b4346b6bb56067a83fea9d1" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 40033 } } }
I20260810 07:24:03.978659 27474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:03.979090 27485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:03.981859 27485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:03.987061 27485 catalog_manager.cc:1347] Generated new cluster ID: 4baefe4ddacf4e328f0ff13a134b9088
I20260810 07:24:03.987129 27485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:03.996102 27485 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:03.997337 27485 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:04.008498 27485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1: Generated new TSK 0
I20260810 07:24:04.009178 27485 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:24:04.048166 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:40033
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:04.162324 27505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:04.162590 27505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:04.162631 27505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:04.166713 27505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:04.166766 27505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:04.166872 27505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:24:04.171957 27505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:40033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:04.173017 27505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:04.174033 27505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:04.183993 27517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:04.183955 27511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:04.183949 27512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:04.184199 27505 server_base.cc:1034] running on GCE node
I20260810 07:24:04.184692 27505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:04.185341 27505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:04.186534 27505 hybrid_clock.cc:648] HybridClock initialized: now 1786346644186515 us; error 30 us; skew 500 ppm
Aug 10 07:24:04 dist-test-slave-44d2 krb5kdc[27332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346644, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:04.189329 27505 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:24:04.190181 27505 webserver.cc:460] Webserver started at http://127.1.125.193:33411/ using document root <none> and password file <none>
I20260810 07:24:04.190752 27505 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:04.190845 27505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:04.191088 27505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:04.193231 27505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "e089f66d639d4062bd2211022ca9b99b"
format_stamp: "Formatted at 2026-08-10 07:24:04 on dist-test-slave-44d2"
server_key: "04d59cc6cd6e9b02462538b9a8363e93"
server_key_iv: "b53c1c15db40bafa56d69cf2833d26cd"
server_key_version: "encryptionkey@0"
I20260810 07:24:04.193804 27505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "e089f66d639d4062bd2211022ca9b99b"
format_stamp: "Formatted at 2026-08-10 07:24:04 on dist-test-slave-44d2"
server_key: "04d59cc6cd6e9b02462538b9a8363e93"
server_key_iv: "b53c1c15db40bafa56d69cf2833d26cd"
server_key_version: "encryptionkey@0"
I20260810 07:24:04.197666 27505 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:24:04.200066 27533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:04.201400 27505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:24:04.201545 27505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "e089f66d639d4062bd2211022ca9b99b"
format_stamp: "Formatted at 2026-08-10 07:24:04 on dist-test-slave-44d2"
server_key: "04d59cc6cd6e9b02462538b9a8363e93"
server_key_iv: "b53c1c15db40bafa56d69cf2833d26cd"
server_key_version: "encryptionkey@0"
I20260810 07:24:04.201664 27505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:04.215873 27505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:04.219404 27505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:04.219640 27505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:04.220263 27505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:04.221204 27505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:04.221254 27505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:04.221326 27505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:04.221356 27505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:04.230367 27505 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:43893
I20260810 07:24:04.230448 27725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:43893 every 8 connection(s)
I20260810 07:24:04.231338 27505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:24:04.235313 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 27505
I20260810 07:24:04.235495 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:24:04.235809 1527 external_mini_cluster.cc:1428] Setting key 2effb6ece744b1286c0f1293821c14b9
Aug 10 07:24:04 dist-test-slave-44d2 krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346644, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:24:04.247219 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.233347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:34923 (local address 127.1.125.254:40033)
0810 07:24:04.233489 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:04.233493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:04.234364 (+ 871us) server_negotiation.cc:407] Connection header received
0810 07:24:04.235787 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:04.235791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:04.235849 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:04.236009 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:04.237459 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.238002 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.239671 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.239934 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.240046 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.243005 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:04.243014 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:04.243017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:04.243043 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:04.244859 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:04.245463 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:04.245471 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:04.245473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:04.245567 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:04.245959 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:04.245965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:04.245967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:04.246179 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:04.246311 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:04.246857 (+ 546us) server_negotiation.cc:299] Negotiation successful
0810 07:24:04.247015 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:24:04.248202 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.233733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40033 (local address 127.1.125.193:34923)
0810 07:24:04.234215 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:04.234258 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:04.235538 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:04.236221 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:04.236230 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:04.236726 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:04.237313 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.237330 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.238251 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.238254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.239423 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.239442 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.240118 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.240124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.240369 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.241104 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:04.241130 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:04.242809 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:04.244997 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:04.245004 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:04.245020 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:04.245325 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:04.245674 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:04.245678 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:04.245680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:04.245839 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:04.246311 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:04.246320 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:04.246596 (+ 276us) client_negotiation.cc:769] Sending connection context
0810 07:24:04.246808 (+ 212us) client_negotiation.cc:240] Negotiation successful
0810 07:24:04.247175 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":118,"threads_started":1}
I20260810 07:24:04.249630 27727 heartbeater.cc:347] Connected to a master server at 127.1.125.254:40033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
I20260810 07:24:04.249926 27727 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:04.250501 27727 heartbeater.cc:511] Master 127.1.125.254:40033 requested a full tablet report, sending...
I20260810 07:24:04.252171 27400 ts_manager.cc:194] Registered new tserver with Master: e089f66d639d4062bd2211022ca9b99b (127.1.125.193:43893)
I20260810 07:24:04.253890 27400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:34923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:24:04.267740 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:40033
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:04.384454 27733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:04.384706 27733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:04.384745 27733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:04.388578 27733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:04.388623 27733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:04.388677 27733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:24:04.393795 27733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:40033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:04.394747 27733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:04.395701 27733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:04.405369 27747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:04.405414 27743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:04.405370 27744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:04.405431 27733 server_base.cc:1034] running on GCE node
I20260810 07:24:04.406037 27733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:04.406697 27733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:04.407858 27733 hybrid_clock.cc:648] HybridClock initialized: now 1786346644407828 us; error 42 us; skew 500 ppm
Aug 10 07:24:04 dist-test-slave-44d2 krb5kdc[27332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346644, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:04.410703 27733 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:24:04.411571 27733 webserver.cc:460] Webserver started at http://127.1.125.194:38629/ using document root <none> and password file <none>
I20260810 07:24:04.412145 27733 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:04.412241 27733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:04.412530 27733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:04.414574 27733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "c9833702afe342d4ae07d61a3e1c2876"
format_stamp: "Formatted at 2026-08-10 07:24:04 on dist-test-slave-44d2"
server_key: "42b63d7590fa4d2d7fff88d7844d7fd3"
server_key_iv: "75ecc8aefe4b148dc3edc51ce6cc84fb"
server_key_version: "encryptionkey@0"
I20260810 07:24:04.415225 27733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "c9833702afe342d4ae07d61a3e1c2876"
format_stamp: "Formatted at 2026-08-10 07:24:04 on dist-test-slave-44d2"
server_key: "42b63d7590fa4d2d7fff88d7844d7fd3"
server_key_iv: "75ecc8aefe4b148dc3edc51ce6cc84fb"
server_key_version: "encryptionkey@0"
I20260810 07:24:04.419217 27733 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:24:04.421838 27760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:04.423002 27733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:24:04.423187 27733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "c9833702afe342d4ae07d61a3e1c2876"
format_stamp: "Formatted at 2026-08-10 07:24:04 on dist-test-slave-44d2"
server_key: "42b63d7590fa4d2d7fff88d7844d7fd3"
server_key_iv: "75ecc8aefe4b148dc3edc51ce6cc84fb"
server_key_version: "encryptionkey@0"
I20260810 07:24:04.423312 27733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:04.442816 27733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:04.445952 27733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:04.446180 27733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:04.446776 27733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:04.447611 27733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:04.447685 27733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:04.447747 27733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:04.447791 27733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:04.457116 27733 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:40453
I20260810 07:24:04.457145 27941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:40453 every 8 connection(s)
I20260810 07:24:04.457998 27733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:24:04.460187 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 27733
I20260810 07:24:04.460340 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:24:04.460729 1527 external_mini_cluster.cc:1428] Setting key 689c175fbad0670755d5a2fdae6755f9
Aug 10 07:24:04 dist-test-slave-44d2 krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346644, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
I20260810 07:24:04.473713 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.459826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:58221 (local address 127.1.125.254:40033)
0810 07:24:04.459983 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:04.459988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:04.461105 (+ 1117us) server_negotiation.cc:407] Connection header received
0810 07:24:04.462453 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:04.462457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:04.462511 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:04.462600 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:04.463773 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.464664 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.466505 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.466735 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.466821 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.469921 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:04.469925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:04.469927 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:04.469951 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:04.471453 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:04.472022 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:04.472027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:04.472029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:04.472115 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:04.472462 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:04.472466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:04.472468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:04.472718 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:04.472833 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:04.473355 (+ 522us) server_negotiation.cc:299] Negotiation successful
0810 07:24:04.473504 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:24:04.474553 27945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.460334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40033 (local address 127.1.125.194:58221)
0810 07:24:04.460921 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:04.460976 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:04.462236 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:04.462739 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:04.462745 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:04.463109 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:04.463613 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.463628 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.464823 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.464828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.466366 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.466381 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.466910 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.466917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.467177 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.467854 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:04.467881 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:04.469737 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:04.471610 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:04.471617 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:04.471634 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:04.471906 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:04.472212 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:04.472216 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:04.472217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:04.472362 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:04.472833 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:04.472841 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:04.473104 (+ 263us) client_negotiation.cc:769] Sending connection context
0810 07:24:04.473299 (+ 195us) client_negotiation.cc:240] Negotiation successful
0810 07:24:04.473642 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":149,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
I20260810 07:24:04.475992 27943 heartbeater.cc:347] Connected to a master server at 127.1.125.254:40033
I20260810 07:24:04.476297 27943 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:04.476895 27943 heartbeater.cc:511] Master 127.1.125.254:40033 requested a full tablet report, sending...
I20260810 07:24:04.478200 27400 ts_manager.cc:194] Registered new tserver with Master: c9833702afe342d4ae07d61a3e1c2876 (127.1.125.194:40453)
I20260810 07:24:04.479008 27400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:58221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:24:04.495157 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:40033
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:04.611996 27949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:04.612262 27949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:04.612308 27949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:04.616288 27949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:04.616344 27949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:04.616401 27949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:24:04.621798 27949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:40033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:04.622787 27949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:04.623754 27949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:04.633445 27955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:04.633458 27956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:04.633458 27961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:04.633700 27949 server_base.cc:1034] running on GCE node
I20260810 07:24:04.634166 27949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:04.634804 27949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:04.635995 27949 hybrid_clock.cc:648] HybridClock initialized: now 1786346644635965 us; error 46 us; skew 500 ppm
Aug 10 07:24:04 dist-test-slave-44d2 krb5kdc[27332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346644, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:04.638906 27949 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:24:04.639775 27949 webserver.cc:460] Webserver started at http://127.1.125.195:35199/ using document root <none> and password file <none>
I20260810 07:24:04.640357 27949 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:04.640451 27949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:04.640733 27949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:04.642787 27949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "e10b4c30fac24b9e8ebcee40c556e982"
format_stamp: "Formatted at 2026-08-10 07:24:04 on dist-test-slave-44d2"
server_key: "8e4f332c582bbda8dd760142568de41c"
server_key_iv: "2f209c590be863fc9211538ba4fb4162"
server_key_version: "encryptionkey@0"
I20260810 07:24:04.643384 27949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "e10b4c30fac24b9e8ebcee40c556e982"
format_stamp: "Formatted at 2026-08-10 07:24:04 on dist-test-slave-44d2"
server_key: "8e4f332c582bbda8dd760142568de41c"
server_key_iv: "2f209c590be863fc9211538ba4fb4162"
server_key_version: "encryptionkey@0"
I20260810 07:24:04.648065 27949 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:24:04.651791 27970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:04.653409 27949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:24:04.653563 27949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "e10b4c30fac24b9e8ebcee40c556e982"
format_stamp: "Formatted at 2026-08-10 07:24:04 on dist-test-slave-44d2"
server_key: "8e4f332c582bbda8dd760142568de41c"
server_key_iv: "2f209c590be863fc9211538ba4fb4162"
server_key_version: "encryptionkey@0"
I20260810 07:24:04.653731 27949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:04.673879 27949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:04.677008 27949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:04.677276 27949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:04.677930 27949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:04.678952 27949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:04.679026 27949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:04.679101 27949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:04.679145 27949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:04.688548 27949 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:35373
I20260810 07:24:04.688572 28155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:35373 every 8 connection(s)
I20260810 07:24:04.689464 27949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:24:04.692708 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 27949
I20260810 07:24:04.692869 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:24:04.693324 1527 external_mini_cluster.cc:1428] Setting key a465190672019782f75c2b687ca7ce36
Aug 10 07:24:04 dist-test-slave-44d2 krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346644, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:24:04.704394 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.691341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:55687 (local address 127.1.125.254:40033)
0810 07:24:04.691509 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:04.691512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:04.692699 (+ 1187us) server_negotiation.cc:407] Connection header received
0810 07:24:04.694727 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:04.694730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:04.694787 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:04.694866 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:04.696373 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.697046 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.698202 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.698445 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.698533 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.701155 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:04.701158 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:04.701160 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:04.701184 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:04.702654 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:04.703117 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:04.703119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:04.703120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:04.703202 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:04.703491 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:04.703493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:04.703494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:04.703663 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:04.703738 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:04.704176 (+ 438us) server_negotiation.cc:299] Negotiation successful
0810 07:24:04.704274 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 07:24:04.705302 28161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.691800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:40033 (local address 127.1.125.195:55687)
0810 07:24:04.692447 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:04.692542 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:04.694515 (+ 1973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:04.695139 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:04.695149 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:04.695583 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:04.696224 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.696241 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.697170 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.697173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.698078 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.698088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.698583 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.698585 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.698735 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.699327 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:04.699344 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:04.700976 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:04.702754 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:04.702758 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:04.702768 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:04.703014 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:04.703283 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:04.703285 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:04.703286 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:04.703411 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:04.703752 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:04.703757 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:04.703964 (+ 207us) client_negotiation.cc:769] Sending connection context
0810 07:24:04.704124 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 07:24:04.704356 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":128,"threads_started":1}
I20260810 07:24:04.706493 28157 heartbeater.cc:347] Connected to a master server at 127.1.125.254:40033
I20260810 07:24:04.706776 28157 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:04.707330 28157 heartbeater.cc:511] Master 127.1.125.254:40033 requested a full tablet report, sending...
I20260810 07:24:04.708398 27400 ts_manager.cc:194] Registered new tserver with Master: e10b4c30fac24b9e8ebcee40c556e982 (127.1.125.195:35373)
I20260810 07:24:04.709079 27400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:55687
I20260810 07:24:04.719375 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:24:04.728600 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.721496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36610 (local address 127.1.125.254:40033)
0810 07:24:04.721617 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:04.721620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:04.721853 (+ 233us) server_negotiation.cc:407] Connection header received
0810 07:24:04.721940 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:04.721942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:04.721979 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:04.722055 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:04.722735 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.723260 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.724051 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.724251 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.724325 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.725689 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:04.725692 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:04.725694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:04.725713 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:04.727002 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:04.727480 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:04.727483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:04.727484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:04.727561 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:04.727801 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:04.727803 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:04.727804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:04.728018 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:04.728118 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:04.728331 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 07:24:04.728423 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:24:04.731297 27400 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:24:04.734596 27400 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:24:04.749511 28174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.743130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:35373 (local address 127.0.0.1:34932)
0810 07:24:04.743832 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:04.743882 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:04.744013 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:04.745103 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:04.745106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:04.745124 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:04.745257 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.745265 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.746796 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.746801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.747754 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.747765 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.748967 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.748970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.749089 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.749132 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 07:24:04.749245 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 07:24:04.749352 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":17,"thread_start_us":76,"threads_started":1}
I20260810 07:24:04.749519 28179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.743319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34932 (local address 127.1.125.195:35373)
0810 07:24:04.744670 (+ 1351us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:04.744681 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:04.744702 (+ 21us) server_negotiation.cc:407] Connection header received
0810 07:24:04.744771 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:04.744776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:04.744948 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:04.745421 (+ 473us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:04.745580 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.746646 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.747908 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.748845 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.748967 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.749155 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:04.749272 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 07:24:04.749366 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1241,"thread_start_us":126,"threads_started":1}
I20260810 07:24:04.750411 28176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.743534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:43893 (local address 127.0.0.1:37498)
0810 07:24:04.744623 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:04.744634 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:04.744748 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:04.746613 (+ 1865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:04.746616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:04.746638 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:04.746737 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.746748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.747871 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.747874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.748715 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.748727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.749999 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.750006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.750110 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.750147 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 07:24:04.750217 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:24:04.750287 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1038,"thread_start_us":89,"threads_started":1}
I20260810 07:24:04.750645 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.743301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:40453 (local address 127.0.0.1:50458)
0810 07:24:04.744290 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:04.744304 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:04.744386 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:04.744969 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:04.744975 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:04.745000 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:04.745128 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.745135 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.746598 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.746601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.748915 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.748929 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.750245 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.750252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.750349 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.750373 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:24:04.750448 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:24:04.750524 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"mutex_wait_us":47,"thread_start_us":115,"threads_started":1}
I20260810 07:24:04.750809 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.743955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.1.125.193:43893)
0810 07:24:04.744426 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:04.744448 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:04.744727 (+ 279us) server_negotiation.cc:407] Connection header received
0810 07:24:04.744884 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:04.744907 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:04.745112 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:04.745263 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:04.746873 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.747737 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.748869 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.749707 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.749802 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.749922 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:04.750572 (+ 650us) server_negotiation.cc:299] Negotiation successful
0810 07:24:04.750669 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":112,"threads_started":1}
I20260810 07:24:04.752980 28049 tablet_service.cc:1467] Processing CreateTablet for tablet 4bd8c77f668a46778c318be8352b39dc (DEFAULT_TABLE table=test-table [id=5d965f7c5d2648d9bb94ba8bb0349025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:04.753286 27620 tablet_service.cc:1467] Processing CreateTablet for tablet 4bd8c77f668a46778c318be8352b39dc (DEFAULT_TABLE table=test-table [id=5d965f7c5d2648d9bb94ba8bb0349025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:04.753863 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.743931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50458 (local address 127.1.125.194:40453)
0810 07:24:04.744435 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:04.744449 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:04.744475 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:24:04.744585 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:04.744591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:04.744805 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:04.744967 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:04.745254 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.746448 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.749055 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.749910 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.750013 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.750176 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:04.750849 (+ 673us) server_negotiation.cc:299] Negotiation successful
0810 07:24:04.753617 (+ 2768us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":151,"threads_started":1}
I20260810 07:24:04.754169 28049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bd8c77f668a46778c318be8352b39dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:04.754360 27620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bd8c77f668a46778c318be8352b39dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:04.756832 27837 tablet_service.cc:1467] Processing CreateTablet for tablet 4bd8c77f668a46778c318be8352b39dc (DEFAULT_TABLE table=test-table [id=5d965f7c5d2648d9bb94ba8bb0349025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:04.757948 27837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bd8c77f668a46778c318be8352b39dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:04.760813 28184 tablet_bootstrap.cc:492] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b: Bootstrap starting.
I20260810 07:24:04.762434 28185 tablet_bootstrap.cc:492] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982: Bootstrap starting.
I20260810 07:24:04.763029 28184 tablet_bootstrap.cc:654] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:04.763774 28184 log.cc:824] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:04.764801 28185 tablet_bootstrap.cc:654] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:04.765829 28184 tablet_bootstrap.cc:492] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b: No bootstrap required, opened a new log
I20260810 07:24:04.766047 28184 ts_tablet_manager.cc:1397] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:24:04.766094 28185 log.cc:824] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:04.766178 28187 tablet_bootstrap.cc:492] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876: Bootstrap starting.
I20260810 07:24:04.767834 28185 tablet_bootstrap.cc:492] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982: No bootstrap required, opened a new log
I20260810 07:24:04.768055 28185 ts_tablet_manager.cc:1397] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:24:04.768867 28187 tablet_bootstrap.cc:654] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:04.769696 28187 log.cc:824] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:04.770687 28184 raft_consensus.cc:348] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10b4c30fac24b9e8ebcee40c556e982" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 35373 } } peers { permanent_uuid: "c9833702afe342d4ae07d61a3e1c2876" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40453 } } peers { permanent_uuid: "e089f66d639d4062bd2211022ca9b99b" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43893 } }
I20260810 07:24:04.770954 28184 raft_consensus.cc:374] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:04.771016 28184 raft_consensus.cc:729] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e089f66d639d4062bd2211022ca9b99b, State: Initialized, Role: FOLLOWER
I20260810 07:24:04.771498 28184 consensus_queue.cc:260] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10b4c30fac24b9e8ebcee40c556e982" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 35373 } } peers { permanent_uuid: "c9833702afe342d4ae07d61a3e1c2876" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40453 } } peers { permanent_uuid: "e089f66d639d4062bd2211022ca9b99b" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43893 } }
I20260810 07:24:04.771714 28187 tablet_bootstrap.cc:492] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876: No bootstrap required, opened a new log
I20260810 07:24:04.771333 28185 raft_consensus.cc:348] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10b4c30fac24b9e8ebcee40c556e982" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 35373 } } peers { permanent_uuid: "c9833702afe342d4ae07d61a3e1c2876" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40453 } } peers { permanent_uuid: "e089f66d639d4062bd2211022ca9b99b" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43893 } }
I20260810 07:24:04.771904 28187 ts_tablet_manager.cc:1397] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 07:24:04.771890 28185 raft_consensus.cc:374] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:04.771963 28185 raft_consensus.cc:729] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e10b4c30fac24b9e8ebcee40c556e982, State: Initialized, Role: FOLLOWER
I20260810 07:24:04.772413 28185 consensus_queue.cc:260] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10b4c30fac24b9e8ebcee40c556e982" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 35373 } } peers { permanent_uuid: "c9833702afe342d4ae07d61a3e1c2876" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40453 } } peers { permanent_uuid: "e089f66d639d4062bd2211022ca9b99b" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43893 } }
I20260810 07:24:04.772571 27727 heartbeater.cc:503] Master 127.1.125.254:40033 was elected leader, sending a full tablet report...
I20260810 07:24:04.773161 28157 heartbeater.cc:503] Master 127.1.125.254:40033 was elected leader, sending a full tablet report...
I20260810 07:24:04.773497 28185 ts_tablet_manager.cc:1428] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:24:04.774230 28184 ts_tablet_manager.cc:1428] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260810 07:24:04.775290 28187 raft_consensus.cc:348] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10b4c30fac24b9e8ebcee40c556e982" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 35373 } } peers { permanent_uuid: "c9833702afe342d4ae07d61a3e1c2876" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40453 } } peers { permanent_uuid: "e089f66d639d4062bd2211022ca9b99b" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43893 } }
I20260810 07:24:04.775566 28187 raft_consensus.cc:374] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:04.775628 28187 raft_consensus.cc:729] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9833702afe342d4ae07d61a3e1c2876, State: Initialized, Role: FOLLOWER
I20260810 07:24:04.776051 28187 consensus_queue.cc:260] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10b4c30fac24b9e8ebcee40c556e982" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 35373 } } peers { permanent_uuid: "c9833702afe342d4ae07d61a3e1c2876" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40453 } } peers { permanent_uuid: "e089f66d639d4062bd2211022ca9b99b" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43893 } }
I20260810 07:24:04.776731 27943 heartbeater.cc:503] Master 127.1.125.254:40033 was elected leader, sending a full tablet report...
I20260810 07:24:04.776993 28187 ts_tablet_manager.cc:1428] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260810 07:24:04.881920 28191 raft_consensus.cc:482] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:24:04.882215 28191 raft_consensus.cc:504] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10b4c30fac24b9e8ebcee40c556e982" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 35373 } } peers { permanent_uuid: "c9833702afe342d4ae07d61a3e1c2876" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40453 } } peers { permanent_uuid: "e089f66d639d4062bd2211022ca9b99b" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43893 } }
I20260810 07:24:04.883533 28191 leader_election.cc:288] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9833702afe342d4ae07d61a3e1c2876 (127.1.125.194:40453), e089f66d639d4062bd2211022ca9b99b (127.1.125.193:43893)
I20260810 07:24:04.887648 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.883867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:34931 (local address 127.1.125.194:40453)
0810 07:24:04.883997 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:04.884003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:04.884088 (+ 85us) server_negotiation.cc:407] Connection header received
0810 07:24:04.884239 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:04.884243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:04.884305 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:04.884405 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:04.884772 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.885367 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.886596 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.887036 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.887131 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.887215 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:04.887427 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 07:24:04.887506 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:24:04.887660 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.883977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:59249 (local address 127.1.125.193:43893)
0810 07:24:04.884147 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:04.884154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:04.884415 (+ 261us) server_negotiation.cc:407] Connection header received
0810 07:24:04.884528 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:04.884531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:04.884582 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:04.884670 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:04.884984 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.885536 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.886657 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.887052 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.887201 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.887258 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:04.887479 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 07:24:04.887554 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 07:24:04.887660 28161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.883785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:40453 (local address 127.1.125.195:34931)
0810 07:24:04.883995 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:04.884011 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:04.884118 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:04.884439 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:04.884444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:04.884467 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:04.884644 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.884655 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.885521 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.885525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.886456 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.886467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.887172 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.887176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.887287 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.887316 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 07:24:04.887380 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:24:04.887475 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":60}
I20260810 07:24:04.887666 28194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.883846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:43893 (local address 127.1.125.195:59249)
0810 07:24:04.884307 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:04.884321 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:04.884420 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:04.884692 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:04.884695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:04.884734 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:04.884865 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.884871 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.885686 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.885689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.886495 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:04.886504 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.887221 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.887226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:04.887324 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.887342 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 07:24:04.887440 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 07:24:04.887523 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":176,"threads_started":1}
I20260810 07:24:04.888386 27868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bd8c77f668a46778c318be8352b39dc" candidate_uuid: "e10b4c30fac24b9e8ebcee40c556e982" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9833702afe342d4ae07d61a3e1c2876" is_pre_election: true
I20260810 07:24:04.888589 27648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bd8c77f668a46778c318be8352b39dc" candidate_uuid: "e10b4c30fac24b9e8ebcee40c556e982" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e089f66d639d4062bd2211022ca9b99b" is_pre_election: true
I20260810 07:24:04.888753 27868 raft_consensus.cc:2454] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e10b4c30fac24b9e8ebcee40c556e982 in term 0.
I20260810 07:24:04.888908 27648 raft_consensus.cc:2454] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e10b4c30fac24b9e8ebcee40c556e982 in term 0.
I20260810 07:24:04.889283 27974 leader_election.cc:302] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9833702afe342d4ae07d61a3e1c2876, e10b4c30fac24b9e8ebcee40c556e982; no voters:
I20260810 07:24:04.889586 28191 raft_consensus.cc:2781] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:24:04.889683 28191 raft_consensus.cc:482] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:24:04.889727 28191 raft_consensus.cc:3037] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:04.890797 28191 raft_consensus.cc:504] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10b4c30fac24b9e8ebcee40c556e982" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 35373 } } peers { permanent_uuid: "c9833702afe342d4ae07d61a3e1c2876" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40453 } } peers { permanent_uuid: "e089f66d639d4062bd2211022ca9b99b" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43893 } }
I20260810 07:24:04.891160 28191 leader_election.cc:288] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [CANDIDATE]: Term 1 election: Requested vote from peers c9833702afe342d4ae07d61a3e1c2876 (127.1.125.194:40453), e089f66d639d4062bd2211022ca9b99b (127.1.125.193:43893)
I20260810 07:24:04.891474 27868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bd8c77f668a46778c318be8352b39dc" candidate_uuid: "e10b4c30fac24b9e8ebcee40c556e982" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9833702afe342d4ae07d61a3e1c2876"
I20260810 07:24:04.891496 27648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bd8c77f668a46778c318be8352b39dc" candidate_uuid: "e10b4c30fac24b9e8ebcee40c556e982" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e089f66d639d4062bd2211022ca9b99b"
I20260810 07:24:04.891608 27648 raft_consensus.cc:3037] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:04.891619 27868 raft_consensus.cc:3037] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:04.893015 27648 raft_consensus.cc:2454] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e10b4c30fac24b9e8ebcee40c556e982 in term 1.
I20260810 07:24:04.893015 27868 raft_consensus.cc:2454] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e10b4c30fac24b9e8ebcee40c556e982 in term 1.
I20260810 07:24:04.893361 27978 leader_election.cc:302] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e089f66d639d4062bd2211022ca9b99b, e10b4c30fac24b9e8ebcee40c556e982; no voters:
I20260810 07:24:04.893528 28191 raft_consensus.cc:2781] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:04.893814 28191 raft_consensus.cc:686] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [term 1 LEADER]: Becoming Leader. State: Replica: e10b4c30fac24b9e8ebcee40c556e982, State: Running, Role: LEADER
I20260810 07:24:04.894177 28191 consensus_queue.cc:237] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10b4c30fac24b9e8ebcee40c556e982" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 35373 } } peers { permanent_uuid: "c9833702afe342d4ae07d61a3e1c2876" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40453 } } peers { permanent_uuid: "e089f66d639d4062bd2211022ca9b99b" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43893 } }
I20260810 07:24:04.897459 27399 catalog_manager.cc:5495] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 reported cstate change: term changed from 0 to 1, leader changed from <none> to e10b4c30fac24b9e8ebcee40c556e982 (127.1.125.195). New cstate: current_term: 1 leader_uuid: "e10b4c30fac24b9e8ebcee40c556e982" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10b4c30fac24b9e8ebcee40c556e982" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 35373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9833702afe342d4ae07d61a3e1c2876" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e089f66d639d4062bd2211022ca9b99b" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43893 } health_report { overall_health: UNKNOWN } } }
I20260810 07:24:04.937806 28179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:04.934060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34938 (local address 127.1.125.195:35373)
0810 07:24:04.934226 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:04.934232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:04.934246 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:24:04.934333 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:04.934335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:04.934390 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:04.934510 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:04.934807 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.935319 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.936220 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:04.936442 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:04.936608 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:04.936946 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:04.937386 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:04.937529 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:04.937593 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 07:24:04.937653 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 07:24:04.940414 28159 tablet.cc:2369] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:24:04.943727 27868 raft_consensus.cc:1261] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876 [term 1 FOLLOWER]: Refusing update from remote peer e10b4c30fac24b9e8ebcee40c556e982: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:04.943755 27648 raft_consensus.cc:1261] T 4bd8c77f668a46778c318be8352b39dc P e089f66d639d4062bd2211022ca9b99b [term 1 FOLLOWER]: Refusing update from remote peer e10b4c30fac24b9e8ebcee40c556e982: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:04.944463 28191 consensus_queue.cc:1035] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9833702afe342d4ae07d61a3e1c2876" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:04.944710 28195 consensus_queue.cc:1035] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e089f66d639d4062bd2211022ca9b99b" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 43893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:04.952775 28202 mvcc.cc:205] Tried to move back new op lower bound from 7316875857682317312 to 7316875857489154048. Current Snapshot: MvccSnapshot[applied={T|T < 7316875857682317312}]
I20260810 07:24:04.954312 28201 mvcc.cc:205] Tried to move back new op lower bound from 7316875857682317312 to 7316875857489154048. Current Snapshot: MvccSnapshot[applied={T|T < 7316875857682317312}]
I20260810 07:24:04.966127 27400 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:24:04.966324 27400 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:24:04.968802 27400 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6dfa57c51b4346b6bb56067a83fea9d1: Sending DeleteTablet for 3 replicas of tablet 4bd8c77f668a46778c318be8352b39dc
I20260810 07:24:04.969632 27837 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bd8c77f668a46778c318be8352b39dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:04 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:50458
I20260810 07:24:04.969702 28049 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bd8c77f668a46778c318be8352b39dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:04 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:34932
I20260810 07:24:04.969727 27620 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bd8c77f668a46778c318be8352b39dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:04 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:37498
I20260810 07:24:04.970242 28215 tablet_replica.cc:330] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876: stopping tablet replica
I20260810 07:24:04.970364 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 27505
I20260810 07:24:04.978178 28215 raft_consensus.cc:2229] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:24:04.978595 28215 raft_consensus.cc:2258] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:24:04.979496 28214 tablet_replica.cc:330] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982: stopping tablet replica
I20260810 07:24:04.979840 28214 raft_consensus.cc:2229] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:24:04.980286 28214 raft_consensus.cc:2258] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:24:04.982669 28214 ts_tablet_manager.cc:1905] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:24:04.986001 28214 ts_tablet_manager.cc:1918] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:24:04.986187 28214 log.cc:1196] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/4bd8c77f668a46778c318be8352b39dc
I20260810 07:24:04.986603 28214 ts_tablet_manager.cc:1939] T 4bd8c77f668a46778c318be8352b39dc P e10b4c30fac24b9e8ebcee40c556e982: Deleting consensus metadata
I20260810 07:24:04.987639 28215 ts_tablet_manager.cc:1905] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:24:04.988073 27379 catalog_manager.cc:4841] TS e10b4c30fac24b9e8ebcee40c556e982 (127.1.125.195:35373): tablet 4bd8c77f668a46778c318be8352b39dc (table test-table [id=5d965f7c5d2648d9bb94ba8bb0349025]) successfully deleted
I20260810 07:24:04.990686 28215 ts_tablet_manager.cc:1918] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:24:04.990818 28215 log.cc:1196] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/4bd8c77f668a46778c318be8352b39dc
I20260810 07:24:04.991111 28215 ts_tablet_manager.cc:1939] T 4bd8c77f668a46778c318be8352b39dc P c9833702afe342d4ae07d61a3e1c2876: Deleting consensus metadata
I20260810 07:24:04.992280 27380 catalog_manager.cc:4841] TS c9833702afe342d4ae07d61a3e1c2876 (127.1.125.194:40453): tablet 4bd8c77f668a46778c318be8352b39dc (table test-table [id=5d965f7c5d2648d9bb94ba8bb0349025]) successfully deleted
W20260810 07:24:04.998245 27381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:43893 (error 108)
I20260810 07:24:04.998423 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 27733
I20260810 07:24:05.004648 28174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.004197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:43893 (local address 127.0.0.1:37514)
0810 07:24:05.004375 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:05.004517 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 07:24:05.004899 27381 catalog_manager.cc:4584] TS e089f66d639d4062bd2211022ca9b99b (127.1.125.193:43893): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4bd8c77f668a46778c318be8352b39dc: Network error: Client connection negotiation failed: client connection to 127.1.125.193:43893: connect: Connection refused (error 111)
I20260810 07:24:05.012992 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 27949
I20260810 07:24:05.023670 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 27351
2026-08-10T07:24:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2739 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T07:24:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:05Z Disabled control of system clock
I20260810 07:24:05.061944 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:42931
--webserver_interface=127.1.125.254
--webserver_port=0
--builtin_ntp_servers=127.1.125.212:38451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:42931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:24:05.174777 28223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:05.175037 28223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:05.175072 28223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:05.178966 28223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:05.179019 28223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:05.179035 28223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:05.179051 28223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:05.179067 28223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:05.183804 28223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:38451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:42931
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:42931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:05.184998 28223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:05.186313 28223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:05.195158 28230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:05.195230 28231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:05.195289 28233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:05.195211 28223 server_base.cc:1034] running on GCE node
I20260810 07:24:05.195888 28223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:05.197094 28223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:05.198293 28223 hybrid_clock.cc:648] HybridClock initialized: now 1786346645198263 us; error 43 us; skew 500 ppm
I20260810 07:24:05.200192 28223 webserver.cc:460] Webserver started at http://127.1.125.254:34347/ using document root <none> and password file <none>
I20260810 07:24:05.200857 28223 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:05.200958 28223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:05.201215 28223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:05.203260 28223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "2d2c24550f764709a9b0fe5ee9e887a8"
format_stamp: "Formatted at 2026-08-10 07:24:05 on dist-test-slave-44d2"
server_key: "7fa751b482b5754f0e43e8c909c7192c"
server_key_iv: "d18682b718d3d43bebb675646877eac0"
server_key_version: "encryptionkey@0"
I20260810 07:24:05.203842 28223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "2d2c24550f764709a9b0fe5ee9e887a8"
format_stamp: "Formatted at 2026-08-10 07:24:05 on dist-test-slave-44d2"
server_key: "7fa751b482b5754f0e43e8c909c7192c"
server_key_iv: "d18682b718d3d43bebb675646877eac0"
server_key_version: "encryptionkey@0"
I20260810 07:24:05.207667 28223 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:24:05.210204 28244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:05.211438 28223 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:24:05.211597 28223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "2d2c24550f764709a9b0fe5ee9e887a8"
format_stamp: "Formatted at 2026-08-10 07:24:05 on dist-test-slave-44d2"
server_key: "7fa751b482b5754f0e43e8c909c7192c"
server_key_iv: "d18682b718d3d43bebb675646877eac0"
server_key_version: "encryptionkey@0"
I20260810 07:24:05.211710 28223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:05.228778 28223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:05.229534 28223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:05.229766 28223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:05.237663 28330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:42931 every 8 connection(s)
I20260810 07:24:05.237668 28223 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:42931
I20260810 07:24:05.238741 28223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:05.242111 28332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:05.247587 28332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8: Bootstrap starting.
I20260810 07:24:05.247933 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 28223
I20260810 07:24:05.248095 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:05.248381 1527 external_mini_cluster.cc:1428] Setting key 558d7b9ea89f5f652469c2e323ed3306
I20260810 07:24:05.250433 28332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:05.251201 28332 log.cc:824] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:05.253396 28332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8: No bootstrap required, opened a new log
I20260810 07:24:05.257824 28332 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2c24550f764709a9b0fe5ee9e887a8" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 42931 } }
I20260810 07:24:05.258101 28332 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:05.258193 28332 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2c24550f764709a9b0fe5ee9e887a8, State: Initialized, Role: FOLLOWER
I20260810 07:24:05.258318 28335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.249762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43674 (local address 127.1.125.254:42931)
0810 07:24:05.250399 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:05.250417 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:05.250453 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:24:05.251117 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:05.251133 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:05.251607 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:05.251989 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:05.252336 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.253369 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.254616 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.255084 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.255209 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.256214 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:05.256219 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:05.256232 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:05.256728 (+ 496us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:05.256785 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:05.256806 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:05.256885 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:05.257076 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 07:24:05.257363 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":160,"threads_started":1}
I20260810 07:24:05.258685 28332 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2c24550f764709a9b0fe5ee9e887a8" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 42931 } }
I20260810 07:24:05.258847 28332 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:05.258939 28332 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:05.259063 28332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:05.260201 28332 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2c24550f764709a9b0fe5ee9e887a8" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 42931 } }
I20260810 07:24:05.260578 28332 leader_election.cc:302] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d2c24550f764709a9b0fe5ee9e887a8; no voters:
I20260810 07:24:05.260960 28332 leader_election.cc:288] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:05.261058 28337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:05.261302 28337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2d2c24550f764709a9b0fe5ee9e887a8, State: Running, Role: LEADER
I20260810 07:24:05.261619 28337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2c24550f764709a9b0fe5ee9e887a8" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 42931 } }
I20260810 07:24:05.262300 28332 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:05.263229 28340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d2c24550f764709a9b0fe5ee9e887a8. Latest consensus state: current_term: 1 leader_uuid: "2d2c24550f764709a9b0fe5ee9e887a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2c24550f764709a9b0fe5ee9e887a8" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 42931 } } }
I20260810 07:24:05.263384 28340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:05.263729 28338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d2c24550f764709a9b0fe5ee9e887a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2c24550f764709a9b0fe5ee9e887a8" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 42931 } } }
I20260810 07:24:05.263827 28338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:05.264106 28353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:05.266819 28353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:05.272429 28353 catalog_manager.cc:1347] Generated new cluster ID: cbd27cdbb110480c99029a4042323cb1
I20260810 07:24:05.272523 28353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:05.324188 28353 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:05.325254 28353 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:05.330909 28353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8: Generated new TSK 0
I20260810 07:24:05.331645 28353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:05.334571 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:42931
--builtin_ntp_servers=127.1.125.212:38451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:24:05.463248 28369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:05.463595 28369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:05.463645 28369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:05.468067 28369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:05.468137 28369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:05.468192 28369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:24:05.472999 28369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:38451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:42931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:05.474023 28369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:05.475353 28369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:05.485399 28379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:05.485387 28378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:05.485391 28381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:05.485643 28369 server_base.cc:1034] running on GCE node
I20260810 07:24:05.486199 28369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:05.486825 28369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:05.488027 28369 hybrid_clock.cc:648] HybridClock initialized: now 1786346645488005 us; error 33 us; skew 500 ppm
I20260810 07:24:05.489964 28369 webserver.cc:460] Webserver started at http://127.1.125.193:39449/ using document root <none> and password file <none>
I20260810 07:24:05.490533 28369 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:05.490594 28369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:05.490769 28369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:05.492923 28369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "b704c55534e5418b910104afdf22144e"
format_stamp: "Formatted at 2026-08-10 07:24:05 on dist-test-slave-44d2"
server_key: "6ae6b3465df7c898a5e6a0c53699a5c6"
server_key_iv: "f2a754075b7e90244a202030ed25f8d9"
server_key_version: "encryptionkey@0"
I20260810 07:24:05.493453 28369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "b704c55534e5418b910104afdf22144e"
format_stamp: "Formatted at 2026-08-10 07:24:05 on dist-test-slave-44d2"
server_key: "6ae6b3465df7c898a5e6a0c53699a5c6"
server_key_iv: "f2a754075b7e90244a202030ed25f8d9"
server_key_version: "encryptionkey@0"
I20260810 07:24:05.497156 28369 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:24:05.499609 28388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:05.501006 28369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:24:05.501132 28369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "b704c55534e5418b910104afdf22144e"
format_stamp: "Formatted at 2026-08-10 07:24:05 on dist-test-slave-44d2"
server_key: "6ae6b3465df7c898a5e6a0c53699a5c6"
server_key_iv: "f2a754075b7e90244a202030ed25f8d9"
server_key_version: "encryptionkey@0"
I20260810 07:24:05.501225 28369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:05.529397 28369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:05.530123 28369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:05.530289 28369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:05.530874 28369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:05.531807 28369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:05.531855 28369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:05.531901 28369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:05.531921 28369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:05.540835 28369 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:35433
I20260810 07:24:05.540863 28575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:35433 every 8 connection(s)
I20260810 07:24:05.541779 28369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:24:05.542686 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 28369
I20260810 07:24:05.542806 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:24:05.543131 1527 external_mini_cluster.cc:1428] Setting key 40cc996c77dde2b28fcc8aef1cb38fec
I20260810 07:24:05.545258 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:42931
--builtin_ntp_servers=127.1.125.212:38451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:24:05.553158 28335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.543882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:56455 (local address 127.1.125.254:42931)
0810 07:24:05.544025 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:05.544030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:05.545345 (+ 1315us) server_negotiation.cc:407] Connection header received
0810 07:24:05.546870 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:05.546876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:05.546942 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:05.547054 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:05.548339 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.548946 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.550853 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.551082 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.551163 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.552605 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:05.552609 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:05.552611 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:05.552624 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:05.552641 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:05.552649 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:05.552704 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:05.552925 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 07:24:05.553045 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:24:05.554448 28578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.544368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:42931 (local address 127.1.125.193:56455)
0810 07:24:05.545196 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:05.545250 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:05.546641 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:05.547307 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:05.547319 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:05.547439 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:05.548179 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:05.548201 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.549107 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.549113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:05.550718 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:05.550735 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.551260 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.551268 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:05.551524 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.552229 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:24:05.552232 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:05.552296 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:24:05.552394 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:05.552744 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:05.552758 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:05.552766 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:24:05.552923 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 07:24:05.553273 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":173,"threads_started":1}
I20260810 07:24:05.555838 28576 heartbeater.cc:347] Connected to a master server at 127.1.125.254:42931
I20260810 07:24:05.556094 28576 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:05.556692 28576 heartbeater.cc:511] Master 127.1.125.254:42931 requested a full tablet report, sending...
I20260810 07:24:05.558238 28271 ts_manager.cc:194] Registered new tserver with Master: b704c55534e5418b910104afdf22144e (127.1.125.193:35433)
I20260810 07:24:05.559412 28271 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.125.193:56455
W20260810 07:24:05.664289 28579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:05.664610 28579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:05.664686 28579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:05.668618 28579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:05.668666 28579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:05.668761 28579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:24:05.673535 28579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:38451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:42931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:05.674553 28579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:05.675923 28579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:05.686039 28588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:05.686148 28592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:05.686040 28589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:05.686264 28579 server_base.cc:1034] running on GCE node
I20260810 07:24:05.686733 28579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:05.687410 28579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:05.688563 28579 hybrid_clock.cc:648] HybridClock initialized: now 1786346645688530 us; error 47 us; skew 500 ppm
I20260810 07:24:05.690376 28579 webserver.cc:460] Webserver started at http://127.1.125.194:41471/ using document root <none> and password file <none>
I20260810 07:24:05.690976 28579 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:05.691073 28579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:05.691318 28579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:05.693517 28579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "8acc775340434451981411868eac42ac"
format_stamp: "Formatted at 2026-08-10 07:24:05 on dist-test-slave-44d2"
server_key: "2015ea28fde754678c1fad40318f5cb2"
server_key_iv: "087b59d1e3b0ec32e6e5044c254712f0"
server_key_version: "encryptionkey@0"
I20260810 07:24:05.694123 28579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "8acc775340434451981411868eac42ac"
format_stamp: "Formatted at 2026-08-10 07:24:05 on dist-test-slave-44d2"
server_key: "2015ea28fde754678c1fad40318f5cb2"
server_key_iv: "087b59d1e3b0ec32e6e5044c254712f0"
server_key_version: "encryptionkey@0"
I20260810 07:24:05.697970 28579 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:24:05.700459 28601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:05.701819 28579 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:24:05.701999 28579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "8acc775340434451981411868eac42ac"
format_stamp: "Formatted at 2026-08-10 07:24:05 on dist-test-slave-44d2"
server_key: "2015ea28fde754678c1fad40318f5cb2"
server_key_iv: "087b59d1e3b0ec32e6e5044c254712f0"
server_key_version: "encryptionkey@0"
I20260810 07:24:05.702155 28579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:05.719328 28579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:05.720005 28579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:05.720217 28579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:05.720906 28579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:05.721868 28579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:05.721961 28579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:05.722054 28579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:05.722107 28579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:05.731251 28579 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:35327
I20260810 07:24:05.731285 28776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:35327 every 8 connection(s)
I20260810 07:24:05.732290 28579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:24:05.733315 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 28579
I20260810 07:24:05.733438 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:24:05.733763 1527 external_mini_cluster.cc:1428] Setting key 0a3fc002d7cd7e4da635876a1ba57698
I20260810 07:24:05.735984 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:42931
--builtin_ntp_servers=127.1.125.212:38451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:24:05.743129 28335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.734399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:55903 (local address 127.1.125.254:42931)
0810 07:24:05.734542 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:05.734546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:05.735769 (+ 1223us) server_negotiation.cc:407] Connection header received
0810 07:24:05.737347 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:05.737351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:05.737408 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:05.737498 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:05.738673 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.739201 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.741069 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.741311 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.741439 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.742620 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:05.742624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:05.742626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:05.742638 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:05.742657 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:05.742665 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:05.742747 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:05.742918 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 07:24:05.742984 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:24:05.743963 28784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.734861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:42931 (local address 127.1.125.194:55903)
0810 07:24:05.735620 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:05.735661 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:05.737136 (+ 1475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:05.737660 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:05.737666 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:05.737774 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:05.738506 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:05.738528 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.739337 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.739344 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:05.740900 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:05.740917 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.741487 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.741492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:05.741668 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.742341 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:24:05.742343 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:05.742383 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:24:05.742446 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:05.742749 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:05.742763 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:05.742770 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 07:24:05.742903 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 07:24:05.743204 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":178,"threads_started":1}
I20260810 07:24:05.745234 28777 heartbeater.cc:347] Connected to a master server at 127.1.125.254:42931
I20260810 07:24:05.745550 28777 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:05.746244 28777 heartbeater.cc:511] Master 127.1.125.254:42931 requested a full tablet report, sending...
I20260810 07:24:05.747453 28271 ts_manager.cc:194] Registered new tserver with Master: 8acc775340434451981411868eac42ac (127.1.125.194:35327)
I20260810 07:24:05.748041 28271 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.125.194:55903
W20260810 07:24:05.854729 28786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:05.854977 28786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:05.855013 28786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:05.858956 28786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:05.859006 28786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:05.859099 28786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:24:05.863909 28786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:38451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:42931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:05.865005 28786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:05.866358 28786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:05.876029 28795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:05.876017 28794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:05.876034 28798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:05.876261 28786 server_base.cc:1034] running on GCE node
I20260810 07:24:05.876824 28786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:05.877518 28786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:05.878726 28786 hybrid_clock.cc:648] HybridClock initialized: now 1786346645878695 us; error 48 us; skew 500 ppm
I20260810 07:24:05.880548 28786 webserver.cc:460] Webserver started at http://127.1.125.195:37709/ using document root <none> and password file <none>
I20260810 07:24:05.881230 28786 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:05.881328 28786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:05.881561 28786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:05.883812 28786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "744b8f44eca54754a08d861c8c8a509f"
format_stamp: "Formatted at 2026-08-10 07:24:05 on dist-test-slave-44d2"
server_key: "af4fc7495424df51aea13de40be74eec"
server_key_iv: "16dd85c6acc85300f1b64ed8f0eb36e9"
server_key_version: "encryptionkey@0"
I20260810 07:24:05.884392 28786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "744b8f44eca54754a08d861c8c8a509f"
format_stamp: "Formatted at 2026-08-10 07:24:05 on dist-test-slave-44d2"
server_key: "af4fc7495424df51aea13de40be74eec"
server_key_iv: "16dd85c6acc85300f1b64ed8f0eb36e9"
server_key_version: "encryptionkey@0"
I20260810 07:24:05.888067 28786 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:24:05.890625 28811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:05.891819 28786 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:24:05.891968 28786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "744b8f44eca54754a08d861c8c8a509f"
format_stamp: "Formatted at 2026-08-10 07:24:05 on dist-test-slave-44d2"
server_key: "af4fc7495424df51aea13de40be74eec"
server_key_iv: "16dd85c6acc85300f1b64ed8f0eb36e9"
server_key_version: "encryptionkey@0"
I20260810 07:24:05.892082 28786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:05.918877 28786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:05.919616 28786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:05.919829 28786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:05.920500 28786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:05.921522 28786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:05.921613 28786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:05.921690 28786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:05.921748 28786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:05.932574 28786 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:34151
I20260810 07:24:05.932591 28975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:34151 every 8 connection(s)
I20260810 07:24:05.933699 28786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:24:05.934630 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 28786
I20260810 07:24:05.934732 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:24:05.935043 1527 external_mini_cluster.cc:1428] Setting key 8565ed637e0ef57b848b17ce21cd64c6
I20260810 07:24:05.943418 28335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.936034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:57821 (local address 127.1.125.254:42931)
0810 07:24:05.936155 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:05.936159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:05.937438 (+ 1279us) server_negotiation.cc:407] Connection header received
0810 07:24:05.938872 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:05.938874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:05.938919 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:05.938982 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:05.939899 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.940424 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.941632 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.941847 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.941929 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.943004 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:05.943007 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:05.943010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:05.943022 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:05.943041 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:05.943049 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:05.943109 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:05.943252 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 07:24:05.943301 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 07:24:05.944300 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.936463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:42931 (local address 127.1.125.195:57821)
0810 07:24:05.937268 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:05.937335 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:05.938678 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:05.939137 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:05.939143 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:05.939229 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:05.939771 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:05.939785 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.940584 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.940598 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:05.941522 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:05.941532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.942003 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.942005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:05.942146 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.942736 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:24:05.942738 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:05.942780 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:24:05.942846 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:05.943127 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:05.943134 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:05.943139 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:24:05.943234 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 07:24:05.943453 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":156,"threads_started":1}
I20260810 07:24:05.945436 28976 heartbeater.cc:347] Connected to a master server at 127.1.125.254:42931
I20260810 07:24:05.945683 28976 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:05.946224 28976 heartbeater.cc:511] Master 127.1.125.254:42931 requested a full tablet report, sending...
I20260810 07:24:05.947252 28271 ts_manager.cc:194] Registered new tserver with Master: 744b8f44eca54754a08d861c8c8a509f (127.1.125.195:34151)
I20260810 07:24:05.947944 28271 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.125.195:57821
I20260810 07:24:05.949286 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:24:05.954862 28335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.951235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43678 (local address 127.1.125.254:42931)
0810 07:24:05.951384 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:05.951389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:05.951619 (+ 230us) server_negotiation.cc:407] Connection header received
0810 07:24:05.951709 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:05.951711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:05.951749 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:05.951818 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:05.952095 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.952697 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.953488 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.953691 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.953766 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.954515 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:05.954518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:05.954520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:05.954531 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:05.954549 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:05.954557 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:05.954619 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:05.954713 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 07:24:05.954761 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 07:24:05.957551 28271 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:24:05.960470 28271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:24:05.974925 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.968825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:34151 (local address 127.0.0.1:57668)
0810 07:24:05.969678 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:05.969727 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:05.969870 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:05.970667 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:05.970673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:05.970695 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:05.970817 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:05.970824 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.971996 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.972003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:05.973209 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:05.973220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.974316 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.974320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:05.974442 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.974487 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 07:24:05.974629 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 07:24:05.974757 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"mutex_wait_us":1,"spinlock_wait_cycles":5376,"thread_start_us":86,"threads_started":1}
I20260810 07:24:05.974956 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.968959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57668 (local address 127.1.125.195:34151)
0810 07:24:05.969413 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:05.969429 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:05.970027 (+ 598us) server_negotiation.cc:407] Connection header received
0810 07:24:05.970115 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:05.970124 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:05.970343 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:05.970494 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:05.970948 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.971877 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.973351 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.974183 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.974295 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.974481 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:05.974663 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 07:24:05.974772 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":105,"threads_started":1}
I20260810 07:24:05.975322 29003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.968958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:35327 (local address 127.0.0.1:34960)
0810 07:24:05.969727 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:05.969740 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:05.969870 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:05.970938 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:05.970941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:05.970960 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:05.971067 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:05.971076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.972542 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.972545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:05.973641 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:05.973651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.974684 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.974687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:05.974770 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.974791 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:24:05.974840 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 07:24:05.974894 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"spinlock_wait_cycles":6656,"thread_start_us":93,"threads_started":1}
I20260810 07:24:05.975689 28997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.969210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:35433 (local address 127.0.0.1:46304)
0810 07:24:05.970096 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:05.970108 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:05.970178 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:05.971139 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:05.971142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:05.971160 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:05.971245 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:05.971250 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.972847 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.972854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:05.973923 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:05.973935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.975010 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.975013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:05.975109 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.975134 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 07:24:05.975189 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 07:24:05.975255 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"thread_start_us":96,"threads_started":1}
I20260810 07:24:05.975875 29005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.969666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46304 (local address 127.1.125.193:35433)
0810 07:24:05.970464 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:05.970480 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:05.970516 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:24:05.970589 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:05.970594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:05.970786 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:05.970965 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:05.971381 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.972531 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.974074 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.974846 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.975408 (+ 562us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.975597 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:05.975644 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 07:24:05.975725 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":112,"threads_started":1}
I20260810 07:24:05.976987 29008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:05.969664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34960 (local address 127.1.125.194:35327)
0810 07:24:05.970495 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:05.970511 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:05.970535 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:24:05.970622 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:05.970629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:05.970794 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:05.970956 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:05.971205 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.972381 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.973769 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:05.974547 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:05.976213 (+ 1666us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:05.976725 (+ 512us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:05.976783 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 07:24:05.976860 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":122,"threads_started":1}
I20260810 07:24:05.978365 28467 tablet_service.cc:1467] Processing CreateTablet for tablet 546b89811f58437fb61af5af339e29d1 (DEFAULT_TABLE table=test-table [id=888acffc6e4f4927bef2edba9ee7b1f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:05.978364 28875 tablet_service.cc:1467] Processing CreateTablet for tablet 546b89811f58437fb61af5af339e29d1 (DEFAULT_TABLE table=test-table [id=888acffc6e4f4927bef2edba9ee7b1f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:05.979537 28467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 546b89811f58437fb61af5af339e29d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:05.979537 28875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 546b89811f58437fb61af5af339e29d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:05.979236 28669 tablet_service.cc:1467] Processing CreateTablet for tablet 546b89811f58437fb61af5af339e29d1 (DEFAULT_TABLE table=test-table [id=888acffc6e4f4927bef2edba9ee7b1f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:05.980299 28669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 546b89811f58437fb61af5af339e29d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:05.986771 29009 tablet_bootstrap.cc:492] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac: Bootstrap starting.
I20260810 07:24:05.987812 29011 tablet_bootstrap.cc:492] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e: Bootstrap starting.
I20260810 07:24:05.988082 29010 tablet_bootstrap.cc:492] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f: Bootstrap starting.
I20260810 07:24:05.989420 29009 tablet_bootstrap.cc:654] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:05.990048 29011 tablet_bootstrap.cc:654] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:05.990048 29010 tablet_bootstrap.cc:654] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:05.990429 29009 log.cc:824] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:05.990898 29010 log.cc:824] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:05.990898 29011 log.cc:824] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:05.992894 29011 tablet_bootstrap.cc:492] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e: No bootstrap required, opened a new log
I20260810 07:24:05.992915 29010 tablet_bootstrap.cc:492] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f: No bootstrap required, opened a new log
I20260810 07:24:05.993124 29011 ts_tablet_manager.cc:1397] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:24:05.993141 29010 ts_tablet_manager.cc:1397] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:24:05.993499 29009 tablet_bootstrap.cc:492] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac: No bootstrap required, opened a new log
I20260810 07:24:05.993701 29009 ts_tablet_manager.cc:1397] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 07:24:05.996769 29011 raft_consensus.cc:348] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744b8f44eca54754a08d861c8c8a509f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34151 } } peers { permanent_uuid: "8acc775340434451981411868eac42ac" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35327 } } peers { permanent_uuid: "b704c55534e5418b910104afdf22144e" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 35433 } }
I20260810 07:24:05.997102 29011 raft_consensus.cc:374] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:05.997175 29011 raft_consensus.cc:729] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b704c55534e5418b910104afdf22144e, State: Initialized, Role: FOLLOWER
I20260810 07:24:05.997536 29010 raft_consensus.cc:348] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744b8f44eca54754a08d861c8c8a509f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34151 } } peers { permanent_uuid: "8acc775340434451981411868eac42ac" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35327 } } peers { permanent_uuid: "b704c55534e5418b910104afdf22144e" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 35433 } }
I20260810 07:24:05.997547 29009 raft_consensus.cc:348] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744b8f44eca54754a08d861c8c8a509f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34151 } } peers { permanent_uuid: "8acc775340434451981411868eac42ac" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35327 } } peers { permanent_uuid: "b704c55534e5418b910104afdf22144e" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 35433 } }
I20260810 07:24:05.997785 29009 raft_consensus.cc:374] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:05.997786 29010 raft_consensus.cc:374] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:05.997835 29009 raft_consensus.cc:729] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8acc775340434451981411868eac42ac, State: Initialized, Role: FOLLOWER
I20260810 07:24:05.997835 29010 raft_consensus.cc:729] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744b8f44eca54754a08d861c8c8a509f, State: Initialized, Role: FOLLOWER
I20260810 07:24:05.997906 29011 consensus_queue.cc:260] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744b8f44eca54754a08d861c8c8a509f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34151 } } peers { permanent_uuid: "8acc775340434451981411868eac42ac" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35327 } } peers { permanent_uuid: "b704c55534e5418b910104afdf22144e" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 35433 } }
I20260810 07:24:05.998256 29009 consensus_queue.cc:260] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744b8f44eca54754a08d861c8c8a509f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34151 } } peers { permanent_uuid: "8acc775340434451981411868eac42ac" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35327 } } peers { permanent_uuid: "b704c55534e5418b910104afdf22144e" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 35433 } }
I20260810 07:24:05.998256 29010 consensus_queue.cc:260] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744b8f44eca54754a08d861c8c8a509f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34151 } } peers { permanent_uuid: "8acc775340434451981411868eac42ac" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35327 } } peers { permanent_uuid: "b704c55534e5418b910104afdf22144e" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 35433 } }
I20260810 07:24:05.998695 28576 heartbeater.cc:503] Master 127.1.125.254:42931 was elected leader, sending a full tablet report...
I20260810 07:24:05.998970 29011 ts_tablet_manager.cc:1428] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:24:05.999199 29009 ts_tablet_manager.cc:1428] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:24:05.999806 28777 heartbeater.cc:503] Master 127.1.125.254:42931 was elected leader, sending a full tablet report...
I20260810 07:24:05.999805 29010 ts_tablet_manager.cc:1428] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 07:24:06.000147 28976 heartbeater.cc:503] Master 127.1.125.254:42931 was elected leader, sending a full tablet report...
W20260810 07:24:06.043088 28577 tablet.cc:2369] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:24:06.080678 29021 raft_consensus.cc:482] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:24:06.080935 29021 raft_consensus.cc:504] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744b8f44eca54754a08d861c8c8a509f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34151 } } peers { permanent_uuid: "8acc775340434451981411868eac42ac" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35327 } } peers { permanent_uuid: "b704c55534e5418b910104afdf22144e" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 35433 } }
I20260810 07:24:06.082321 29021 leader_election.cc:288] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8acc775340434451981411868eac42ac (127.1.125.194:35327), b704c55534e5418b910104afdf22144e (127.1.125.193:35433)
I20260810 07:24:06.086273 29008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:06.082651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:36673 (local address 127.1.125.194:35327)
0810 07:24:06.082779 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:06.082786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:06.083151 (+ 365us) server_negotiation.cc:407] Connection header received
0810 07:24:06.083216 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:06.083219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:06.083278 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:06.083352 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:06.083657 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.084175 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.085288 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.085714 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.085847 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:06.085898 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:06.086099 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 07:24:06.086172 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:24:06.086273 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:06.082539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:35327 (local address 127.1.125.195:36673)
0810 07:24:06.082913 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:06.082930 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:06.083036 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:06.083359 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:06.083362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:06.083375 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:06.083537 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:06.083543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.084296 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.084299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:06.085158 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:06.085168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.085843 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.085846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:06.085953 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:06.085972 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:24:06.086058 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 07:24:06.086133 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":2}
I20260810 07:24:06.086431 29005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:06.082916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:51295 (local address 127.1.125.193:35433)
0810 07:24:06.083069 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:06.083073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:06.083195 (+ 122us) server_negotiation.cc:407] Connection header received
0810 07:24:06.083298 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:06.083300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:06.083344 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:06.083438 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:06.083794 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.084306 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.085420 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.085836 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.085962 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:06.086009 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:06.086213 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 07:24:06.086280 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:24:06.086458 29024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:06.082643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:35433 (local address 127.1.125.195:51295)
0810 07:24:06.083093 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:06.083106 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:06.083213 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:06.083469 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:06.083472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:06.083492 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:06.083672 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:06.083678 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.084421 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.084426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:06.085294 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:06.085303 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.085961 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.085965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:06.086066 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:06.086095 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 07:24:06.086194 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 07:24:06.086265 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":44,"thread_start_us":165,"threads_started":1}
I20260810 07:24:06.087011 28704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546b89811f58437fb61af5af339e29d1" candidate_uuid: "744b8f44eca54754a08d861c8c8a509f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8acc775340434451981411868eac42ac" is_pre_election: true
I20260810 07:24:06.087023 28497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546b89811f58437fb61af5af339e29d1" candidate_uuid: "744b8f44eca54754a08d861c8c8a509f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b704c55534e5418b910104afdf22144e" is_pre_election: true
I20260810 07:24:06.087342 28497 raft_consensus.cc:2454] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 744b8f44eca54754a08d861c8c8a509f in term 0.
I20260810 07:24:06.087347 28704 raft_consensus.cc:2454] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 744b8f44eca54754a08d861c8c8a509f in term 0.
I20260810 07:24:06.087925 28816 leader_election.cc:302] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 744b8f44eca54754a08d861c8c8a509f, 8acc775340434451981411868eac42ac, b704c55534e5418b910104afdf22144e; no voters:
I20260810 07:24:06.088259 29021 raft_consensus.cc:2781] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:24:06.088351 29021 raft_consensus.cc:482] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:24:06.088397 29021 raft_consensus.cc:3037] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:06.089557 29021 raft_consensus.cc:504] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744b8f44eca54754a08d861c8c8a509f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34151 } } peers { permanent_uuid: "8acc775340434451981411868eac42ac" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35327 } } peers { permanent_uuid: "b704c55534e5418b910104afdf22144e" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 35433 } }
I20260810 07:24:06.089926 29021 leader_election.cc:288] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [CANDIDATE]: Term 1 election: Requested vote from peers 8acc775340434451981411868eac42ac (127.1.125.194:35327), b704c55534e5418b910104afdf22144e (127.1.125.193:35433)
I20260810 07:24:06.090278 28704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546b89811f58437fb61af5af339e29d1" candidate_uuid: "744b8f44eca54754a08d861c8c8a509f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8acc775340434451981411868eac42ac"
I20260810 07:24:06.090270 28497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546b89811f58437fb61af5af339e29d1" candidate_uuid: "744b8f44eca54754a08d861c8c8a509f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b704c55534e5418b910104afdf22144e"
I20260810 07:24:06.090405 28704 raft_consensus.cc:3037] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:06.090405 28497 raft_consensus.cc:3037] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:06.091763 28497 raft_consensus.cc:2454] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 744b8f44eca54754a08d861c8c8a509f in term 1.
I20260810 07:24:06.091763 28704 raft_consensus.cc:2454] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 744b8f44eca54754a08d861c8c8a509f in term 1.
I20260810 07:24:06.092108 28815 leader_election.cc:302] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 744b8f44eca54754a08d861c8c8a509f, 8acc775340434451981411868eac42ac, b704c55534e5418b910104afdf22144e; no voters:
I20260810 07:24:06.092326 29021 raft_consensus.cc:2781] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:06.092557 29021 raft_consensus.cc:686] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [term 1 LEADER]: Becoming Leader. State: Replica: 744b8f44eca54754a08d861c8c8a509f, State: Running, Role: LEADER
I20260810 07:24:06.092909 29021 consensus_queue.cc:237] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744b8f44eca54754a08d861c8c8a509f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34151 } } peers { permanent_uuid: "8acc775340434451981411868eac42ac" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35327 } } peers { permanent_uuid: "b704c55534e5418b910104afdf22144e" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 35433 } }
I20260810 07:24:06.096129 28270 catalog_manager.cc:5495] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f reported cstate change: term changed from 0 to 1, leader changed from <none> to 744b8f44eca54754a08d861c8c8a509f (127.1.125.195). New cstate: current_term: 1 leader_uuid: "744b8f44eca54754a08d861c8c8a509f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744b8f44eca54754a08d861c8c8a509f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 34151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8acc775340434451981411868eac42ac" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b704c55534e5418b910104afdf22144e" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 35433 } health_report { overall_health: UNKNOWN } } }
I20260810 07:24:06.126201 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:06.122624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57684 (local address 127.1.125.195:34151)
0810 07:24:06.122788 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:06.122792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:06.122804 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:24:06.122880 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:06.122882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:06.122934 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:06.123051 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:06.123299 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.123802 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.124682 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:06.124909 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:06.125030 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:06.125354 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:06.125769 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:06.125902 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:06.125966 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 07:24:06.126029 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 07:24:06.132181 28497 raft_consensus.cc:1261] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e [term 1 FOLLOWER]: Refusing update from remote peer 744b8f44eca54754a08d861c8c8a509f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:06.132182 28704 raft_consensus.cc:1261] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac [term 1 FOLLOWER]: Refusing update from remote peer 744b8f44eca54754a08d861c8c8a509f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:06.133000 29021 consensus_queue.cc:1035] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8acc775340434451981411868eac42ac" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:06.133177 29025 consensus_queue.cc:1035] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b704c55534e5418b910104afdf22144e" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:06.142413 29032 mvcc.cc:205] Tried to move back new op lower bound from 7316875862550081536 to 7316875862399275008. Current Snapshot: MvccSnapshot[applied={T|T < 7316875862550081536}]
I20260810 07:24:06.142848 29031 mvcc.cc:205] Tried to move back new op lower bound from 7316875862550081536 to 7316875862399275008. Current Snapshot: MvccSnapshot[applied={T|T < 7316875862550081536}]
I20260810 07:24:06.149736 28269 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:24:06.149914 28269 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:24:06.152462 28269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d2c24550f764709a9b0fe5ee9e887a8: Sending DeleteTablet for 3 replicas of tablet 546b89811f58437fb61af5af339e29d1
I20260810 07:24:06.153345 28875 tablet_service.cc:1514] Processing DeleteTablet for tablet 546b89811f58437fb61af5af339e29d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:06 UTC) from {username='slave'} at 127.0.0.1:57668
I20260810 07:24:06.153347 28669 tablet_service.cc:1514] Processing DeleteTablet for tablet 546b89811f58437fb61af5af339e29d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:06 UTC) from {username='slave'} at 127.0.0.1:34960
I20260810 07:24:06.153863 29043 tablet_replica.cc:330] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac: stopping tablet replica
I20260810 07:24:06.153851 28467 tablet_service.cc:1514] Processing DeleteTablet for tablet 546b89811f58437fb61af5af339e29d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:06 UTC) from {username='slave'} at 127.0.0.1:46304
I20260810 07:24:06.154172 29044 tablet_replica.cc:330] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f: stopping tablet replica
I20260810 07:24:06.154300 29045 tablet_replica.cc:330] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e: stopping tablet replica
I20260810 07:24:06.154610 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 28369
I20260810 07:24:06.154618 29045 raft_consensus.cc:2229] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:24:06.154935 29045 raft_consensus.cc:2258] T 546b89811f58437fb61af5af339e29d1 P b704c55534e5418b910104afdf22144e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:24:06.170795 29044 raft_consensus.cc:2229] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:24:06.154639 29043 raft_consensus.cc:2229] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:24:06.171252 29043 raft_consensus.cc:2258] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:24:06.171252 29044 raft_consensus.cc:2258] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:24:06.173130 29043 ts_tablet_manager.cc:1905] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:24:06.173130 29044 ts_tablet_manager.cc:1905] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:24:06.176011 29043 ts_tablet_manager.cc:1918] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:24:06.176009 29044 ts_tablet_manager.cc:1918] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:24:06.176119 29044 log.cc:1196] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/546b89811f58437fb61af5af339e29d1
I20260810 07:24:06.176246 29043 log.cc:1196] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/546b89811f58437fb61af5af339e29d1
I20260810 07:24:06.176468 29044 ts_tablet_manager.cc:1939] T 546b89811f58437fb61af5af339e29d1 P 744b8f44eca54754a08d861c8c8a509f: Deleting consensus metadata
I20260810 07:24:06.176752 29043 ts_tablet_manager.cc:1939] T 546b89811f58437fb61af5af339e29d1 P 8acc775340434451981411868eac42ac: Deleting consensus metadata
I20260810 07:24:06.177867 28247 catalog_manager.cc:4841] TS 8acc775340434451981411868eac42ac (127.1.125.194:35327): tablet 546b89811f58437fb61af5af339e29d1 (table test-table [id=888acffc6e4f4927bef2edba9ee7b1f6]) successfully deleted
I20260810 07:24:06.177867 28246 catalog_manager.cc:4841] TS 744b8f44eca54754a08d861c8c8a509f (127.1.125.195:34151): tablet 546b89811f58437fb61af5af339e29d1 (table test-table [id=888acffc6e4f4927bef2edba9ee7b1f6]) successfully deleted
W20260810 07:24:06.178825 28248 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:35433 (error 108)
I20260810 07:24:06.179769 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 28579
I20260810 07:24:06.179983 28997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:06.179655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:35433 (local address 127.0.0.1:46320)
0810 07:24:06.179820 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:06.179896 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:35433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260810 07:24:06.186681 28248 catalog_manager.cc:4584] TS b704c55534e5418b910104afdf22144e (127.1.125.193:35433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 546b89811f58437fb61af5af339e29d1: Network error: Client connection negotiation failed: client connection to 127.1.125.193:35433: connect: Connection refused (error 111)
I20260810 07:24:06.194780 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 28786
I20260810 07:24:06.207904 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 28223
2026-08-10T07:24:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1183 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:06 dist-test-slave-44d2 krb5kdc[29053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:06 dist-test-slave-44d2 krb5kdc[29053](info): set up 2 sockets
Aug 10 07:24:06 dist-test-slave-44d2 krb5kdc[29053](info): commencing operation
krb5kdc: starting...
W20260810 07:24:07.479653 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:07 dist-test-slave-44d2 krb5kdc[29053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:24:07.607924 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:43115
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:36497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:43115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:07.723073 29074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:07.723335 29074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:07.723376 29074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:07.727378 29074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:07.727437 29074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:07.727458 29074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:07.727474 29074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:07.727489 29074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:07.732697 29074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:36497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:43115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:43115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:07.733664 29074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:07.734638 29074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:24:07.743242 29074 server_base.cc:1034] running on GCE node
W20260810 07:24:07.743135 29081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:07.743137 29084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:07.743137 29082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:07.744014 29074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:07.745144 29074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:07.746302 29074 hybrid_clock.cc:648] HybridClock initialized: now 1786346647746277 us; error 38 us; skew 500 ppm
Aug 10 07:24:07 dist-test-slave-44d2 krb5kdc[29053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346647, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:07.753762 29074 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:24:07.754638 29074 webserver.cc:460] Webserver started at http://127.1.125.254:41579/ using document root <none> and password file <none>
I20260810 07:24:07.755175 29074 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:07.755234 29074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:07.755400 29074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:07.757555 29074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "bb6cd02cc6d84c108c79df390d068e44"
format_stamp: "Formatted at 2026-08-10 07:24:07 on dist-test-slave-44d2"
server_key: "d14690280d109b155b368659c048ca1d"
server_key_iv: "f89087ae7da353febb0a2853c30e835f"
server_key_version: "encryptionkey@0"
I20260810 07:24:07.758103 29074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "bb6cd02cc6d84c108c79df390d068e44"
format_stamp: "Formatted at 2026-08-10 07:24:07 on dist-test-slave-44d2"
server_key: "d14690280d109b155b368659c048ca1d"
server_key_iv: "f89087ae7da353febb0a2853c30e835f"
server_key_version: "encryptionkey@0"
I20260810 07:24:07.761859 29074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:24:07.764179 29097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:07.765460 29074 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:24:07.765595 29074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "bb6cd02cc6d84c108c79df390d068e44"
format_stamp: "Formatted at 2026-08-10 07:24:07 on dist-test-slave-44d2"
server_key: "d14690280d109b155b368659c048ca1d"
server_key_iv: "f89087ae7da353febb0a2853c30e835f"
server_key_version: "encryptionkey@0"
I20260810 07:24:07.765698 29074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:07.785770 29074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:07.789144 29074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:07.789361 29074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:07.796445 29074 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:43115
I20260810 07:24:07.796553 29178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:43115 every 8 connection(s)
I20260810 07:24:07.797462 29074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:07.800411 29179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:07.805238 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 29074
I20260810 07:24:07.805398 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:07.805711 1527 external_mini_cluster.cc:1428] Setting key fb6cba02273ab13f711cac73ea62e037
I20260810 07:24:07.806433 29179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44: Bootstrap starting.
I20260810 07:24:07.809022 29179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:07.809854 29179 log.cc:824] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:07.811791 29179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44: No bootstrap required, opened a new log
Aug 10 07:24:07 dist-test-slave-44d2 krb5kdc[29053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:24:07.815488 29179 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6cd02cc6d84c108c79df390d068e44" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43115 } }
I20260810 07:24:07.815754 29179 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:07.815846 29179 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb6cd02cc6d84c108c79df390d068e44, State: Initialized, Role: FOLLOWER
I20260810 07:24:07.816335 29179 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6cd02cc6d84c108c79df390d068e44" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43115 } }
I20260810 07:24:07.816514 29179 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:07.816615 29179 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:07.816738 29179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:07.817641 29179 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6cd02cc6d84c108c79df390d068e44" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43115 } }
I20260810 07:24:07.817991 29179 leader_election.cc:302] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb6cd02cc6d84c108c79df390d068e44; no voters:
I20260810 07:24:07.818334 29179 leader_election.cc:288] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:07.818405 29185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:07.818629 29185 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [term 1 LEADER]: Becoming Leader. State: Replica: bb6cd02cc6d84c108c79df390d068e44, State: Running, Role: LEADER
I20260810 07:24:07.818988 29185 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6cd02cc6d84c108c79df390d068e44" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43115 } }
I20260810 07:24:07.819693 29179 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:07.820223 29185 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb6cd02cc6d84c108c79df390d068e44. Latest consensus state: current_term: 1 leader_uuid: "bb6cd02cc6d84c108c79df390d068e44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6cd02cc6d84c108c79df390d068e44" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43115 } } }
I20260810 07:24:07.820401 29185 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:07.820770 29186 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb6cd02cc6d84c108c79df390d068e44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6cd02cc6d84c108c79df390d068e44" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43115 } } }
I20260810 07:24:07.820956 29186 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:07.821540 29183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:07.807144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46980 (local address 127.1.125.254:43115)
0810 07:24:07.807580 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:07.807596 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:07.807633 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:24:07.808266 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:07.808283 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:07.808655 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:07.809022 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:07.809861 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:07.810886 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:07.811766 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:07.812301 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:07.812491 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:07.815084 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:07.815089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:07.815104 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:07.815130 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:07.817236 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:07.817702 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:07.817708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:07.817714 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:07.817890 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:07.818212 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:07.818216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:07.818218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:07.818778 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:07.818970 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:07.819430 (+ 460us) server_negotiation.cc:299] Negotiation successful
0810 07:24:07.819921 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":131,"threads_started":1}
W20260810 07:24:07.824973 29205 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:07.825058 29205 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:07.825126 29192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:07.826018 29192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:07.831677 29192 catalog_manager.cc:1347] Generated new cluster ID: c6b8edd6f9c04c1c9ab6fc6ca7a5e4bf
I20260810 07:24:07.831749 29192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:07.848835 29192 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:07.849782 29192 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:07.860014 29192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44: Generated new TSK 0
I20260810 07:24:07.860738 29192 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:24:07.891675 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:43115
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:36497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:08.011215 29219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:08.011447 29219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:08.011487 29219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:08.015487 29219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:08.015545 29219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:08.015632 29219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:24:08.021126 29219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:36497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:43115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:08.022406 29219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:08.023631 29219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:08.038511 29228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:08.038498 29226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:08.038746 29219 server_base.cc:1034] running on GCE node
W20260810 07:24:08.038501 29224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:08.039290 29219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:08.039992 29219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:08.041204 29219 hybrid_clock.cc:648] HybridClock initialized: now 1786346648041187 us; error 25 us; skew 500 ppm
Aug 10 07:24:08 dist-test-slave-44d2 krb5kdc[29053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346648, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:08.043985 29219 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:24:08.044907 29219 webserver.cc:460] Webserver started at http://127.1.125.193:38169/ using document root <none> and password file <none>
I20260810 07:24:08.045492 29219 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:08.045559 29219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:08.045809 29219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:08.047906 29219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "9c2e581b84ef432da582db96153ef494"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-44d2"
server_key: "b0e9759f52ba84d51da390eca0dea0ac"
server_key_iv: "445ba789378e710ef6acac3d2c8b1c60"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.048547 29219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "9c2e581b84ef432da582db96153ef494"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-44d2"
server_key: "b0e9759f52ba84d51da390eca0dea0ac"
server_key_iv: "445ba789378e710ef6acac3d2c8b1c60"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.052201 29219 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:24:08.054672 29242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:08.055868 29219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:24:08.055997 29219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "9c2e581b84ef432da582db96153ef494"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-44d2"
server_key: "b0e9759f52ba84d51da390eca0dea0ac"
server_key_iv: "445ba789378e710ef6acac3d2c8b1c60"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.056113 29219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:08.071030 29219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:08.074131 29219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:08.074358 29219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:08.074936 29219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:08.075764 29219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:08.075839 29219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:08.075911 29219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:08.075965 29219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:08.084889 29219 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:41633
I20260810 07:24:08.084913 29424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:41633 every 8 connection(s)
I20260810 07:24:08.085765 29219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:24:08.089141 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 29219
I20260810 07:24:08.089299 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:24:08.089699 1527 external_mini_cluster.cc:1428] Setting key 9ac35fb57890aeff3789bac68af48a86
Aug 10 07:24:08 dist-test-slave-44d2 krb5kdc[29053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346648, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:24:08.101634 29183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.087781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:57171 (local address 127.1.125.254:43115)
0810 07:24:08.087931 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:08.087936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:08.088863 (+ 927us) server_negotiation.cc:407] Connection header received
0810 07:24:08.090552 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:08.090555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:08.090631 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:08.090725 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:08.092239 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.092836 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.094288 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.094711 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.094836 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.097350 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:08.097357 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:08.097361 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:08.097396 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:08.099176 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:08.099686 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:08.099691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:08.099693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:08.099793 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:08.100361 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:08.100366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:08.100371 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:08.100662 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:08.100788 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:08.101247 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 07:24:08.101419 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:24:08.102317 29427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.088210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:43115 (local address 127.1.125.193:57171)
0810 07:24:08.088718 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:08.088760 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:08.090328 (+ 1568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:08.090874 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:08.090880 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:08.091375 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:08.092077 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.092100 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.092967 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.092974 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.094161 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.094172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.094874 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.094876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.095018 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.095618 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:08.095633 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:08.097172 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:08.099291 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:08.099296 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:08.099307 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:08.099573 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:08.099901 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:08.099904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:08.099905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:08.100232 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:08.100780 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:08.100785 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:08.100998 (+ 213us) client_negotiation.cc:769] Sending connection context
0810 07:24:08.101159 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 07:24:08.101440 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":112,"threads_started":1}
I20260810 07:24:08.103657 29425 heartbeater.cc:347] Connected to a master server at 127.1.125.254:43115
I20260810 07:24:08.103914 29425 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
I20260810 07:24:08.104447 29425 heartbeater.cc:511] Master 127.1.125.254:43115 requested a full tablet report, sending...
I20260810 07:24:08.106108 29122 ts_manager.cc:194] Registered new tserver with Master: 9c2e581b84ef432da582db96153ef494 (127.1.125.193:41633)
I20260810 07:24:08.107290 29122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:57171
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:24:08.121807 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:43115
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:36497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:08.237883 29430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:08.238174 29430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:08.238251 29430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:08.242209 29430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:08.242295 29430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:08.242385 29430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:24:08.247530 29430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:36497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:43115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:08.248613 29430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:08.249665 29430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:08.259717 29442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:08.259724 29440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:08.259727 29439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:08.260116 29430 server_base.cc:1034] running on GCE node
I20260810 07:24:08.260565 29430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:08.261332 29430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:08.262526 29430 hybrid_clock.cc:648] HybridClock initialized: now 1786346648262508 us; error 27 us; skew 500 ppm
Aug 10 07:24:08 dist-test-slave-44d2 krb5kdc[29053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346648, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:08.265410 29430 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:24:08.266335 29430 webserver.cc:460] Webserver started at http://127.1.125.194:40149/ using document root <none> and password file <none>
I20260810 07:24:08.266909 29430 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:08.266973 29430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:08.267268 29430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:08.269467 29430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-44d2"
server_key: "5dac26c45bd6cbd8856994cd471f6180"
server_key_iv: "47b5ee452439f10cfe61d9496a2c67ac"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.270056 29430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-44d2"
server_key: "5dac26c45bd6cbd8856994cd471f6180"
server_key_iv: "47b5ee452439f10cfe61d9496a2c67ac"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.273824 29430 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:24:08.276242 29453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:08.277562 29430 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:24:08.277702 29430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-44d2"
server_key: "5dac26c45bd6cbd8856994cd471f6180"
server_key_iv: "47b5ee452439f10cfe61d9496a2c67ac"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.277813 29430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:08.293022 29430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:08.295835 29430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:08.296047 29430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:08.296692 29430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:08.297618 29430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:08.297699 29430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:08.297773 29430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:08.297811 29430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:08.307041 29430 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:40261
I20260810 07:24:08.307070 29623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:40261 every 8 connection(s)
I20260810 07:24:08.307942 29430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:24:08.309378 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 29430
I20260810 07:24:08.309527 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:24:08.309875 1527 external_mini_cluster.cc:1428] Setting key 77860cee71fce1f2af43bee76d354baa
Aug 10 07:24:08 dist-test-slave-44d2 krb5kdc[29053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346648, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
I20260810 07:24:08.322868 29183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.309888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:58961 (local address 127.1.125.254:43115)
0810 07:24:08.310048 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:08.310053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:08.311065 (+ 1012us) server_negotiation.cc:407] Connection header received
0810 07:24:08.312393 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:08.312400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:08.312465 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:08.312577 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:08.313760 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.314303 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.315527 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.315769 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.315885 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.318709 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:08.318712 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:08.318714 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:08.318737 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:08.320399 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:08.321097 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:08.321101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:08.321104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:08.321217 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:08.321611 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:08.321616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:08.321618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:08.321804 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:08.321914 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:08.322574 (+ 660us) server_negotiation.cc:299] Negotiation successful
0810 07:24:08.322698 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 07:24:08.323750 29631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.310332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:43115 (local address 127.1.125.194:58961)
0810 07:24:08.310899 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:08.310952 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:08.312180 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:08.312773 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:08.312780 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:08.313135 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:08.313610 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.313624 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.314439 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.314443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.315375 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.315385 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.315925 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.315929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.316098 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.316774 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:08.316791 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:08.318536 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:08.320544 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:08.320549 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:08.320564 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:08.320998 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:08.321317 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:08.321323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:08.321327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:08.321511 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:08.321914 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:08.321922 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:08.322268 (+ 346us) client_negotiation.cc:769] Sending connection context
0810 07:24:08.322528 (+ 260us) client_negotiation.cc:240] Negotiation successful
0810 07:24:08.322844 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":192,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
I20260810 07:24:08.324996 29627 heartbeater.cc:347] Connected to a master server at 127.1.125.254:43115
I20260810 07:24:08.325258 29627 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:08.325850 29627 heartbeater.cc:511] Master 127.1.125.254:43115 requested a full tablet report, sending...
I20260810 07:24:08.326980 29122 ts_manager.cc:194] Registered new tserver with Master: 1b95cdd9f5a8486facc28d23c8ec1f7b (127.1.125.194:40261)
I20260810 07:24:08.327854 29122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:58961
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:24:08.339620 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:43115
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:36497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:08.458256 29635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:08.458504 29635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:08.458547 29635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:08.462663 29635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:08.462723 29635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:08.462779 29635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:24:08.468181 29635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:36497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:43115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:08.469367 29635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:08.470430 29635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:08.480237 29644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:08.480341 29648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:08.480307 29643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:08.480520 29635 server_base.cc:1034] running on GCE node
I20260810 07:24:08.481030 29635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:08.481726 29635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:08.482901 29635 hybrid_clock.cc:648] HybridClock initialized: now 1786346648482882 us; error 39 us; skew 500 ppm
Aug 10 07:24:08 dist-test-slave-44d2 krb5kdc[29053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346648, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:08.485754 29635 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:24:08.486673 29635 webserver.cc:460] Webserver started at http://127.1.125.195:41657/ using document root <none> and password file <none>
I20260810 07:24:08.487354 29635 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:08.487422 29635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:08.487665 29635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:08.489845 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "19b7c43784474b1d986e79f09b5de5b4"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-44d2"
server_key: "bcb5570a7cc8ab15ec94044bbc4c56f6"
server_key_iv: "2cbaef70bb74b38ec3190233d2b47c43"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.490450 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "19b7c43784474b1d986e79f09b5de5b4"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-44d2"
server_key: "bcb5570a7cc8ab15ec94044bbc4c56f6"
server_key_iv: "2cbaef70bb74b38ec3190233d2b47c43"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.494328 29635 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:24:08.496853 29656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:08.498090 29635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:24:08.498220 29635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "19b7c43784474b1d986e79f09b5de5b4"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-44d2"
server_key: "bcb5570a7cc8ab15ec94044bbc4c56f6"
server_key_iv: "2cbaef70bb74b38ec3190233d2b47c43"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.498324 29635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:08.509927 29635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:08.512796 29635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:08.513001 29635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:08.513619 29635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:08.514462 29635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:08.514521 29635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:08.514619 29635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:08.514660 29635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:08.524447 29635 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:42851
I20260810 07:24:08.524471 29830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:42851 every 8 connection(s)
I20260810 07:24:08.525401 29635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:24:08.526648 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 29635
I20260810 07:24:08.526795 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:24:08.527108 1527 external_mini_cluster.cc:1428] Setting key 969f7d2056e2813fc6be2e6196667cdc
Aug 10 07:24:08 dist-test-slave-44d2 krb5kdc[29053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346648, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:24:08.538905 29183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.527418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:53969 (local address 127.1.125.254:43115)
0810 07:24:08.527567 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:08.527571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:08.528739 (+ 1168us) server_negotiation.cc:407] Connection header received
0810 07:24:08.530193 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:08.530196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:08.530243 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:08.530313 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:08.531439 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.531961 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.533137 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.533350 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.533431 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.535798 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:08.535801 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:08.535803 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:08.535828 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:08.537147 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:08.537629 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:08.537631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:08.537632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:08.537707 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:08.537998 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:08.538000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:08.538001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:08.538168 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:08.538243 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:08.538690 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 07:24:08.538785 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 07:24:08.539739 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.527922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:43115 (local address 127.1.125.195:53969)
0810 07:24:08.528567 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:08.528628 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:08.530002 (+ 1374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:08.530483 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:08.530502 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:08.530854 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:08.531314 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.531327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.532067 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.532069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.533029 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.533039 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.533477 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.533480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.533631 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.534174 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:08.534190 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:08.535644 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:08.537253 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:08.537258 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:08.537270 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:08.537537 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:08.537786 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:08.537788 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:08.537790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:08.537920 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:08.538254 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:08.538259 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:08.538471 (+ 212us) client_negotiation.cc:769] Sending connection context
0810 07:24:08.538638 (+ 167us) client_negotiation.cc:240] Negotiation successful
0810 07:24:08.538875 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":159,"threads_started":1}
I20260810 07:24:08.540953 29831 heartbeater.cc:347] Connected to a master server at 127.1.125.254:43115
I20260810 07:24:08.541230 29831 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:08.541743 29831 heartbeater.cc:511] Master 127.1.125.254:43115 requested a full tablet report, sending...
I20260810 07:24:08.542778 29121 ts_manager.cc:194] Registered new tserver with Master: 19b7c43784474b1d986e79f09b5de5b4 (127.1.125.195:42851)
I20260810 07:24:08.543344 29121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:53969
I20260810 07:24:08.553371 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:24:08.563891 29183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.555446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46996 (local address 127.1.125.254:43115)
0810 07:24:08.555550 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:08.555554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:08.555836 (+ 282us) server_negotiation.cc:407] Connection header received
0810 07:24:08.555932 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:08.555934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:08.555975 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:08.556042 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:08.556755 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.557260 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.558053 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.558256 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.558330 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.559940 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:08.559943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:08.559945 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:08.559968 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:08.561694 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:08.562418 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:08.562421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:08.562423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:08.562519 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:08.562932 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:08.562935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:08.562937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:08.563149 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:08.563314 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:08.563624 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 07:24:08.563720 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 07:24:08.567101 29121 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:24:08.570554 29121 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:24:08.585229 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.579197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:41633 (local address 127.0.0.1:49188)
0810 07:24:08.579740 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:08.579793 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:08.579925 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:08.580605 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:08.580612 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:08.580642 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:08.580796 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.580807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.582340 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.582345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.583695 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.583706 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.584648 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.584652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.584775 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.584818 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 07:24:08.584942 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 07:24:08.585070 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":99,"threads_started":1}
I20260810 07:24:08.585240 29850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.579354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49188 (local address 127.1.125.193:41633)
0810 07:24:08.579981 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:08.580023 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:08.580050 (+ 27us) server_negotiation.cc:407] Connection header received
0810 07:24:08.580151 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:08.580157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:08.580377 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:08.580972 (+ 595us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:08.581171 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.582209 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.583833 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.584523 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.584640 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.584794 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:08.584982 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 07:24:08.585081 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":70,"threads_started":1}
I20260810 07:24:08.585669 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.579411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:40261 (local address 127.0.0.1:56688)
0810 07:24:08.580569 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:08.580597 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:08.580722 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:08.581269 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:08.581272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:08.581292 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:08.581386 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.581394 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.582813 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.582817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.583924 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.583932 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.585221 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.585225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.585327 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.585360 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 07:24:08.585443 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 07:24:08.585517 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1091,"mutex_wait_us":162,"thread_start_us":88,"threads_started":1}
I20260810 07:24:08.586067 29855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.579991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:42851 (local address 127.0.0.1:60202)
0810 07:24:08.581021 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:08.581034 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:08.581126 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:08.581835 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:08.581838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:08.581857 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:08.582000 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.582007 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.583269 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.583275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.584398 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.584409 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.585655 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.585659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.585759 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.585790 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 07:24:08.585862 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 07:24:08.585929 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"thread_start_us":109,"threads_started":1}
I20260810 07:24:08.586099 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.579932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56688 (local address 127.1.125.194:40261)
0810 07:24:08.580324 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:08.580335 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:08.580821 (+ 486us) server_negotiation.cc:407] Connection header received
0810 07:24:08.580911 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:08.580918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:08.581119 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:08.581281 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:08.581496 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.582668 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.584040 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.584937 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.585030 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.585158 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:08.585894 (+ 736us) server_negotiation.cc:299] Negotiation successful
0810 07:24:08.585989 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":167,"threads_started":1}
I20260810 07:24:08.586531 29856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.580449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60202 (local address 127.1.125.195:42851)
0810 07:24:08.581387 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:08.581399 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:08.581421 (+ 22us) server_negotiation.cc:407] Connection header received
0810 07:24:08.581496 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:08.581501 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:08.581689 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:08.581840 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:08.582129 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.583081 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.584561 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.585375 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.585468 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.585588 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:08.586294 (+ 706us) server_negotiation.cc:299] Negotiation successful
0810 07:24:08.586397 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":823,"thread_start_us":152,"threads_started":1}
I20260810 07:24:08.588762 29326 tablet_service.cc:1467] Processing CreateTablet for tablet ea856615d5b44c6780d05f7303b430b4 (DEFAULT_TABLE table=test-table [id=e32b4cdaf0264e618ed6c20eb0a88c4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:08.589946 29326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea856615d5b44c6780d05f7303b430b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:08.589555 29733 tablet_service.cc:1467] Processing CreateTablet for tablet ea856615d5b44c6780d05f7303b430b4 (DEFAULT_TABLE table=test-table [id=e32b4cdaf0264e618ed6c20eb0a88c4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:08.590034 29522 tablet_service.cc:1467] Processing CreateTablet for tablet ea856615d5b44c6780d05f7303b430b4 (DEFAULT_TABLE table=test-table [id=e32b4cdaf0264e618ed6c20eb0a88c4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:08.590582 29733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea856615d5b44c6780d05f7303b430b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:08.590969 29522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea856615d5b44c6780d05f7303b430b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:08.597203 29858 tablet_bootstrap.cc:492] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b: Bootstrap starting.
I20260810 07:24:08.598344 29861 tablet_bootstrap.cc:492] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4: Bootstrap starting.
I20260810 07:24:08.599448 29858 tablet_bootstrap.cc:654] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:08.600102 29861 tablet_bootstrap.cc:654] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:08.600699 29858 log.cc:824] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:08.600879 29859 tablet_bootstrap.cc:492] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494: Bootstrap starting.
I20260810 07:24:08.601344 29861 log.cc:824] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:08.602943 29858 tablet_bootstrap.cc:492] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b: No bootstrap required, opened a new log
I20260810 07:24:08.603193 29859 tablet_bootstrap.cc:654] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:08.603194 29858 ts_tablet_manager.cc:1397] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:24:08.603839 29861 tablet_bootstrap.cc:492] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4: No bootstrap required, opened a new log
I20260810 07:24:08.603865 29859 log.cc:824] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:08.604029 29861 ts_tablet_manager.cc:1397] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:24:08.605652 29859 tablet_bootstrap.cc:492] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494: No bootstrap required, opened a new log
I20260810 07:24:08.605852 29859 ts_tablet_manager.cc:1397] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:24:08.607400 29858 raft_consensus.cc:348] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2e581b84ef432da582db96153ef494" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 41633 } } peers { permanent_uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40261 } } peers { permanent_uuid: "19b7c43784474b1d986e79f09b5de5b4" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 42851 } }
I20260810 07:24:08.607717 29858 raft_consensus.cc:374] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:08.607815 29858 raft_consensus.cc:729] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b95cdd9f5a8486facc28d23c8ec1f7b, State: Initialized, Role: FOLLOWER
I20260810 07:24:08.607895 29861 raft_consensus.cc:348] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2e581b84ef432da582db96153ef494" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 41633 } } peers { permanent_uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40261 } } peers { permanent_uuid: "19b7c43784474b1d986e79f09b5de5b4" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 42851 } }
I20260810 07:24:08.608165 29861 raft_consensus.cc:374] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:08.608224 29861 raft_consensus.cc:729] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b7c43784474b1d986e79f09b5de5b4, State: Initialized, Role: FOLLOWER
I20260810 07:24:08.608333 29858 consensus_queue.cc:260] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2e581b84ef432da582db96153ef494" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 41633 } } peers { permanent_uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40261 } } peers { permanent_uuid: "19b7c43784474b1d986e79f09b5de5b4" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 42851 } }
I20260810 07:24:08.608666 29861 consensus_queue.cc:260] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2e581b84ef432da582db96153ef494" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 41633 } } peers { permanent_uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40261 } } peers { permanent_uuid: "19b7c43784474b1d986e79f09b5de5b4" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 42851 } }
I20260810 07:24:08.609469 29858 ts_tablet_manager.cc:1428] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260810 07:24:08.609378 29859 raft_consensus.cc:348] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2e581b84ef432da582db96153ef494" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 41633 } } peers { permanent_uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40261 } } peers { permanent_uuid: "19b7c43784474b1d986e79f09b5de5b4" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 42851 } }
I20260810 07:24:08.609642 29861 ts_tablet_manager.cc:1428] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:24:08.609702 29859 raft_consensus.cc:374] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:08.609803 29859 raft_consensus.cc:729] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2e581b84ef432da582db96153ef494, State: Initialized, Role: FOLLOWER
I20260810 07:24:08.609915 29627 heartbeater.cc:503] Master 127.1.125.254:43115 was elected leader, sending a full tablet report...
I20260810 07:24:08.609992 29831 heartbeater.cc:503] Master 127.1.125.254:43115 was elected leader, sending a full tablet report...
I20260810 07:24:08.610256 29859 consensus_queue.cc:260] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2e581b84ef432da582db96153ef494" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 41633 } } peers { permanent_uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40261 } } peers { permanent_uuid: "19b7c43784474b1d986e79f09b5de5b4" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 42851 } }
I20260810 07:24:08.610872 29425 heartbeater.cc:503] Master 127.1.125.254:43115 was elected leader, sending a full tablet report...
I20260810 07:24:08.611218 29859 ts_tablet_manager.cc:1428] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:24:08.619504 29876 raft_consensus.cc:482] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:24:08.619696 29876 raft_consensus.cc:504] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2e581b84ef432da582db96153ef494" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 41633 } } peers { permanent_uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40261 } } peers { permanent_uuid: "19b7c43784474b1d986e79f09b5de5b4" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 42851 } }
I20260810 07:24:08.620959 29876 leader_election.cc:288] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b95cdd9f5a8486facc28d23c8ec1f7b (127.1.125.194:40261), 19b7c43784474b1d986e79f09b5de5b4 (127.1.125.195:42851)
I20260810 07:24:08.624953 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.621324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:58471 (local address 127.1.125.194:40261)
0810 07:24:08.621440 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:08.621446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:08.621676 (+ 230us) server_negotiation.cc:407] Connection header received
0810 07:24:08.621825 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:08.621828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:08.621881 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:08.621964 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:08.622319 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.622833 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.623931 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.624351 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.624520 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.624579 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:08.624767 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 07:24:08.624834 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:24:08.624980 29877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.621232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:40261 (local address 127.1.125.193:58471)
0810 07:24:08.621584 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:08.621596 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:08.621720 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:08.622025 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:08.622028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:08.622050 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:08.622208 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.622213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.622946 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.622951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.623805 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.623815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.624513 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.624517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.624614 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.624637 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:24:08.624748 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 07:24:08.624825 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":33,"thread_start_us":163,"threads_started":1}
I20260810 07:24:08.625191 29856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.621411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:50531 (local address 127.1.125.195:42851)
0810 07:24:08.621542 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:08.621548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:08.621955 (+ 407us) server_negotiation.cc:407] Connection header received
0810 07:24:08.622094 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:08.622099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:08.622147 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:08.622245 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:08.622522 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.623017 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.624125 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.624622 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.624752 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.624808 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:08.625009 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 07:24:08.625079 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:24:08.625213 29879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.621237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:42851 (local address 127.1.125.193:50531)
0810 07:24:08.621862 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:08.621875 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:08.621980 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:08.622256 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:08.622259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:08.622277 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:08.622404 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.622410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.623142 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.623145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.623992 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:08.624001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.624754 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.624758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:08.624882 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.624901 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:24:08.624991 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 07:24:08.625058 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":327,"thread_start_us":108,"threads_started":1}
I20260810 07:24:08.625684 29556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea856615d5b44c6780d05f7303b430b4" candidate_uuid: "9c2e581b84ef432da582db96153ef494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b" is_pre_election: true
I20260810 07:24:08.625982 29556 raft_consensus.cc:2454] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2e581b84ef432da582db96153ef494 in term 0.
I20260810 07:24:08.626008 29762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea856615d5b44c6780d05f7303b430b4" candidate_uuid: "9c2e581b84ef432da582db96153ef494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b7c43784474b1d986e79f09b5de5b4" is_pre_election: true
I20260810 07:24:08.626271 29762 raft_consensus.cc:2454] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2e581b84ef432da582db96153ef494 in term 0.
I20260810 07:24:08.626498 29243 leader_election.cc:302] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b95cdd9f5a8486facc28d23c8ec1f7b, 9c2e581b84ef432da582db96153ef494; no voters:
I20260810 07:24:08.626820 29876 raft_consensus.cc:2781] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:24:08.626899 29876 raft_consensus.cc:482] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:24:08.626929 29876 raft_consensus.cc:3037] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:08.627966 29876 raft_consensus.cc:504] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2e581b84ef432da582db96153ef494" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 41633 } } peers { permanent_uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40261 } } peers { permanent_uuid: "19b7c43784474b1d986e79f09b5de5b4" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 42851 } }
I20260810 07:24:08.628332 29876 leader_election.cc:288] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [CANDIDATE]: Term 1 election: Requested vote from peers 1b95cdd9f5a8486facc28d23c8ec1f7b (127.1.125.194:40261), 19b7c43784474b1d986e79f09b5de5b4 (127.1.125.195:42851)
I20260810 07:24:08.628628 29556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea856615d5b44c6780d05f7303b430b4" candidate_uuid: "9c2e581b84ef432da582db96153ef494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b"
I20260810 07:24:08.628782 29556 raft_consensus.cc:3037] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:08.628793 29762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea856615d5b44c6780d05f7303b430b4" candidate_uuid: "9c2e581b84ef432da582db96153ef494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b7c43784474b1d986e79f09b5de5b4"
I20260810 07:24:08.628922 29762 raft_consensus.cc:3037] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:08.629899 29556 raft_consensus.cc:2454] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2e581b84ef432da582db96153ef494 in term 1.
I20260810 07:24:08.629994 29762 raft_consensus.cc:2454] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2e581b84ef432da582db96153ef494 in term 1.
I20260810 07:24:08.630228 29243 leader_election.cc:302] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b95cdd9f5a8486facc28d23c8ec1f7b, 9c2e581b84ef432da582db96153ef494; no voters:
I20260810 07:24:08.630427 29876 raft_consensus.cc:2781] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:08.630637 29876 raft_consensus.cc:686] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2e581b84ef432da582db96153ef494, State: Running, Role: LEADER
I20260810 07:24:08.630956 29876 consensus_queue.cc:237] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2e581b84ef432da582db96153ef494" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 41633 } } peers { permanent_uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40261 } } peers { permanent_uuid: "19b7c43784474b1d986e79f09b5de5b4" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 42851 } }
I20260810 07:24:08.634138 29121 catalog_manager.cc:5495] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2e581b84ef432da582db96153ef494 (127.1.125.193). New cstate: current_term: 1 leader_uuid: "9c2e581b84ef432da582db96153ef494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2e581b84ef432da582db96153ef494" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b7c43784474b1d986e79f09b5de5b4" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260810 07:24:08.650508 29850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.645103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49200 (local address 127.1.125.193:41633)
0810 07:24:08.645230 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:08.645234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:08.645246 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:24:08.645410 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:08.645413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:08.645482 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:08.645618 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:08.646019 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.646888 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.648379 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.648828 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.648956 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.649416 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:08.649986 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:08.650143 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:08.650307 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 07:24:08.650348 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:24:08.657313 29762 raft_consensus.cc:1261] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4 [term 1 FOLLOWER]: Refusing update from remote peer 9c2e581b84ef432da582db96153ef494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:08.657313 29556 raft_consensus.cc:1261] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b [term 1 FOLLOWER]: Refusing update from remote peer 9c2e581b84ef432da582db96153ef494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:08.658046 29880 consensus_queue.cc:1035] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b7c43784474b1d986e79f09b5de5b4" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:08.658248 29876 consensus_queue.cc:1035] T ea856615d5b44c6780d05f7303b430b4 P 9c2e581b84ef432da582db96153ef494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b95cdd9f5a8486facc28d23c8ec1f7b" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 40261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:08.675765 29121 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:24:08.675926 29121 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:24:08.678504 29121 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb6cd02cc6d84c108c79df390d068e44: Sending DeleteTablet for 3 replicas of tablet ea856615d5b44c6780d05f7303b430b4
I20260810 07:24:08.679330 29326 tablet_service.cc:1514] Processing DeleteTablet for tablet ea856615d5b44c6780d05f7303b430b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:08 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:49188
I20260810 07:24:08.679491 29522 tablet_service.cc:1514] Processing DeleteTablet for tablet ea856615d5b44c6780d05f7303b430b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:08 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:56688
I20260810 07:24:08.679948 29733 tablet_service.cc:1514] Processing DeleteTablet for tablet ea856615d5b44c6780d05f7303b430b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:08 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:60202
I20260810 07:24:08.680171 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 29219
I20260810 07:24:08.685807 29893 tablet_replica.cc:330] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b: stopping tablet replica
I20260810 07:24:08.686182 29893 raft_consensus.cc:2229] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:24:08.686481 29893 raft_consensus.cc:2258] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:24:08.689675 29893 ts_tablet_manager.cc:1905] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:24:08.692252 29893 ts_tablet_manager.cc:1918] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:24:08.692420 29893 log.cc:1196] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/ea856615d5b44c6780d05f7303b430b4
I20260810 07:24:08.692829 29894 tablet_replica.cc:330] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4: stopping tablet replica
I20260810 07:24:08.692902 29893 ts_tablet_manager.cc:1939] T ea856615d5b44c6780d05f7303b430b4 P 1b95cdd9f5a8486facc28d23c8ec1f7b: Deleting consensus metadata
I20260810 07:24:08.693171 29894 raft_consensus.cc:2229] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:24:08.693482 29894 raft_consensus.cc:2258] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:24:08.694268 29098 catalog_manager.cc:4841] TS 1b95cdd9f5a8486facc28d23c8ec1f7b (127.1.125.194:40261): tablet ea856615d5b44c6780d05f7303b430b4 (table test-table [id=e32b4cdaf0264e618ed6c20eb0a88c4b]) successfully deleted
I20260810 07:24:08.694901 29894 ts_tablet_manager.cc:1905] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:24:08.697017 29894 ts_tablet_manager.cc:1918] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:24:08.697135 29894 log.cc:1196] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/ea856615d5b44c6780d05f7303b430b4
I20260810 07:24:08.697453 29894 ts_tablet_manager.cc:1939] T ea856615d5b44c6780d05f7303b430b4 P 19b7c43784474b1d986e79f09b5de5b4: Deleting consensus metadata
I20260810 07:24:08.698482 29100 catalog_manager.cc:4841] TS 19b7c43784474b1d986e79f09b5de5b4 (127.1.125.195:42851): tablet ea856615d5b44c6780d05f7303b430b4 (table test-table [id=e32b4cdaf0264e618ed6c20eb0a88c4b]) successfully deleted
W20260810 07:24:08.699942 29101 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:41633 (error 108)
I20260810 07:24:08.700866 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 29430
I20260810 07:24:08.701094 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.700771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:41633 (local address 127.0.0.1:49212)
0810 07:24:08.700910 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:08.700994 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 07:24:08.703981 29101 catalog_manager.cc:4584] TS 9c2e581b84ef432da582db96153ef494 (127.1.125.193:41633): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea856615d5b44c6780d05f7303b430b4: Network error: Client connection negotiation failed: client connection to 127.1.125.193:41633: connect: Connection refused (error 111)
I20260810 07:24:08.712643 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 29635
I20260810 07:24:08.726648 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 29074
2026-08-10T07:24:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2519 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T07:24:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:08Z Disabled control of system clock
I20260810 07:24:08.765240 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:46513
--webserver_interface=127.1.125.254
--webserver_port=0
--builtin_ntp_servers=127.1.125.212:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:46513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:24:08.877761 29904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:08.878021 29904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:08.878054 29904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:08.881913 29904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:08.881969 29904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:08.881984 29904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:08.882001 29904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:08.882016 29904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:08.886718 29904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:46513
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:46513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:08.887768 29904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:08.889292 29904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:08.897912 29912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:08.897965 29904 server_base.cc:1034] running on GCE node
W20260810 07:24:08.897975 29911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:08.898049 29914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:08.898710 29904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:08.899823 29904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:08.901037 29904 hybrid_clock.cc:648] HybridClock initialized: now 1786346648901002 us; error 31 us; skew 500 ppm
I20260810 07:24:08.903071 29904 webserver.cc:460] Webserver started at http://127.1.125.254:43341/ using document root <none> and password file <none>
I20260810 07:24:08.903676 29904 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:08.903777 29904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:08.904019 29904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:08.906159 29904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "3e0d8f1bbfa7458caba14214da2db2d4"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-44d2"
server_key: "a0f7a61bfad8be13868364e573abeda7"
server_key_iv: "352a0ccfed7348f8c947e14cca84b514"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.906744 29904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "3e0d8f1bbfa7458caba14214da2db2d4"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-44d2"
server_key: "a0f7a61bfad8be13868364e573abeda7"
server_key_iv: "352a0ccfed7348f8c947e14cca84b514"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.910660 29904 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:24:08.913108 29925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:08.914345 29904 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:24:08.914508 29904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "3e0d8f1bbfa7458caba14214da2db2d4"
format_stamp: "Formatted at 2026-08-10 07:24:08 on dist-test-slave-44d2"
server_key: "a0f7a61bfad8be13868364e573abeda7"
server_key_iv: "352a0ccfed7348f8c947e14cca84b514"
server_key_version: "encryptionkey@0"
I20260810 07:24:08.914628 29904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:08.934459 29904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:08.935173 29904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:08.935384 29904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:08.943991 29904 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:46513
I20260810 07:24:08.944023 30011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:46513 every 8 connection(s)
I20260810 07:24:08.945027 29904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:08.948210 30012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:08.950799 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 29904
I20260810 07:24:08.950946 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:08.951295 1527 external_mini_cluster.cc:1428] Setting key 8add8c31d0f29439aca94ecf5981c78d
I20260810 07:24:08.954867 30012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4: Bootstrap starting.
I20260810 07:24:08.957254 30012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:08.958024 30012 log.cc:824] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:08.959967 30012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4: No bootstrap required, opened a new log
I20260810 07:24:08.960779 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:08.952815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39970 (local address 127.1.125.254:46513)
0810 07:24:08.953468 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:08.953489 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:08.953523 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:24:08.954174 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:08.954192 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:08.954630 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:08.954978 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:08.955429 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.956402 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.957348 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:08.957859 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:08.958049 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:08.958968 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:08.958973 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:08.958986 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:08.959381 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:08.959441 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:08.959461 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:08.959579 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:08.959699 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 07:24:08.959958 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":143,"threads_started":1}
I20260810 07:24:08.963903 30012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0d8f1bbfa7458caba14214da2db2d4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46513 } }
I20260810 07:24:08.964151 30012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:08.964244 30012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e0d8f1bbfa7458caba14214da2db2d4, State: Initialized, Role: FOLLOWER
I20260810 07:24:08.964717 30012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0d8f1bbfa7458caba14214da2db2d4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46513 } }
I20260810 07:24:08.964872 30012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:08.964960 30012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:08.965072 30012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:08.965952 30012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0d8f1bbfa7458caba14214da2db2d4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46513 } }
I20260810 07:24:08.966298 30012 leader_election.cc:302] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e0d8f1bbfa7458caba14214da2db2d4; no voters:
I20260810 07:24:08.966620 30012 leader_election.cc:288] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:08.966698 30018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:08.966962 30018 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [term 1 LEADER]: Becoming Leader. State: Replica: 3e0d8f1bbfa7458caba14214da2db2d4, State: Running, Role: LEADER
I20260810 07:24:08.967289 30018 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0d8f1bbfa7458caba14214da2db2d4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46513 } }
I20260810 07:24:08.967963 30012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:08.969224 30019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e0d8f1bbfa7458caba14214da2db2d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0d8f1bbfa7458caba14214da2db2d4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46513 } } }
I20260810 07:24:08.969224 30020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e0d8f1bbfa7458caba14214da2db2d4. Latest consensus state: current_term: 1 leader_uuid: "3e0d8f1bbfa7458caba14214da2db2d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0d8f1bbfa7458caba14214da2db2d4" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 46513 } } }
I20260810 07:24:08.969391 30019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:08.969391 30020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:08.969918 30029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:08.973031 30029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:08.978644 30029 catalog_manager.cc:1347] Generated new cluster ID: c140a909393244b786176a3189f3208c
I20260810 07:24:08.978711 30029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:08.995728 30029 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:08.996742 30029 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:09.007269 30029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4: Generated new TSK 0
I20260810 07:24:09.008002 30029 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:09.016748 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46513
--builtin_ntp_servers=127.1.125.212:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:24:09.136701 30043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:09.137023 30043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:09.137105 30043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:09.141201 30043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:09.141279 30043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:09.141369 30043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:24:09.146066 30043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:09.147116 30043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:09.148581 30043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:09.158228 30055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:09.158244 30056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:09.158229 30060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:09.158578 30043 server_base.cc:1034] running on GCE node
I20260810 07:24:09.159119 30043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:09.159824 30043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:09.161034 30043 hybrid_clock.cc:648] HybridClock initialized: now 1786346649161011 us; error 46 us; skew 500 ppm
I20260810 07:24:09.162989 30043 webserver.cc:460] Webserver started at http://127.1.125.193:37187/ using document root <none> and password file <none>
I20260810 07:24:09.163597 30043 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:09.163698 30043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:09.163981 30043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:09.166134 30043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "6c46a55ecb4640888ae43be7702bbef6"
format_stamp: "Formatted at 2026-08-10 07:24:09 on dist-test-slave-44d2"
server_key: "2b33113d6c552dc8ae47b19295628e76"
server_key_iv: "7c1b21cb50edfda7fe52f066cdf86898"
server_key_version: "encryptionkey@0"
I20260810 07:24:09.166712 30043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "6c46a55ecb4640888ae43be7702bbef6"
format_stamp: "Formatted at 2026-08-10 07:24:09 on dist-test-slave-44d2"
server_key: "2b33113d6c552dc8ae47b19295628e76"
server_key_iv: "7c1b21cb50edfda7fe52f066cdf86898"
server_key_version: "encryptionkey@0"
I20260810 07:24:09.170737 30043 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:24:09.173151 30069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:09.174405 30043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:24:09.174553 30043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "6c46a55ecb4640888ae43be7702bbef6"
format_stamp: "Formatted at 2026-08-10 07:24:09 on dist-test-slave-44d2"
server_key: "2b33113d6c552dc8ae47b19295628e76"
server_key_iv: "7c1b21cb50edfda7fe52f066cdf86898"
server_key_version: "encryptionkey@0"
I20260810 07:24:09.174679 30043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:09.193419 30043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:09.194180 30043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:09.194453 30043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:09.195127 30043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:09.196292 30043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:09.196377 30043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:09.196460 30043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:09.196578 30043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260810 07:24:09.206698 30043 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:46469
I20260810 07:24:09.206768 30228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:46469 every 8 connection(s)
I20260810 07:24:09.207886 30043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:24:09.215052 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 30043
I20260810 07:24:09.215171 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:24:09.215485 1527 external_mini_cluster.cc:1428] Setting key 01193b17467f07e2846d9bb8bf48a45c
I20260810 07:24:09.217658 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.209827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:43807 (local address 127.1.125.254:46513)
0810 07:24:09.210051 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:09.210054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:09.211117 (+ 1063us) server_negotiation.cc:407] Connection header received
0810 07:24:09.212575 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:09.212579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:09.212643 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:09.212735 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:09.213623 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.214182 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.215387 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.215650 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.215766 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.217099 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:09.217102 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:09.217104 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:09.217159 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:09.217186 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:09.217199 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:09.217278 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:09.217443 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 07:24:09.217510 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260810 07:24:09.217960 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46513
--builtin_ntp_servers=127.1.125.212:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:24:09.218709 30232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.210191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:46513 (local address 127.1.125.193:43807)
0810 07:24:09.210928 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:09.210981 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:09.212292 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:09.212900 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:09.212907 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:09.213003 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:09.213481 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.213493 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.214305 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.214308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.215258 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.215269 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.215822 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.215827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.216063 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.216765 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:24:09.216767 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:09.216822 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:24:09.216910 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:09.217291 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:09.217300 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:09.217305 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:24:09.217408 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 07:24:09.217756 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":125,"threads_started":1}
I20260810 07:24:09.219893 30229 heartbeater.cc:347] Connected to a master server at 127.1.125.254:46513
I20260810 07:24:09.220139 30229 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:09.220714 30229 heartbeater.cc:511] Master 127.1.125.254:46513 requested a full tablet report, sending...
I20260810 07:24:09.222206 29950 ts_manager.cc:194] Registered new tserver with Master: 6c46a55ecb4640888ae43be7702bbef6 (127.1.125.193:46469)
I20260810 07:24:09.223337 29950 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.125.193:43807
W20260810 07:24:09.337451 30233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:09.337750 30233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:09.337824 30233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:09.341814 30233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:09.341894 30233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:09.341986 30233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:24:09.346750 30233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:09.347819 30233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:09.349267 30233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:09.359170 30246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:09.359190 30243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:09.359218 30242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:09.359371 30233 server_base.cc:1034] running on GCE node
I20260810 07:24:09.359876 30233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:09.360563 30233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:09.361737 30233 hybrid_clock.cc:648] HybridClock initialized: now 1786346649361722 us; error 24 us; skew 500 ppm
I20260810 07:24:09.363628 30233 webserver.cc:460] Webserver started at http://127.1.125.194:45971/ using document root <none> and password file <none>
I20260810 07:24:09.364214 30233 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:09.364280 30233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:09.364609 30233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:09.366704 30233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "fe921b351de843138c03547374f56080"
format_stamp: "Formatted at 2026-08-10 07:24:09 on dist-test-slave-44d2"
server_key: "5d27df9fb1b87f68e50a8cd8b07ddeca"
server_key_iv: "fd6eae0f9ffdec44edd04a4a643ca0df"
server_key_version: "encryptionkey@0"
I20260810 07:24:09.367260 30233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "fe921b351de843138c03547374f56080"
format_stamp: "Formatted at 2026-08-10 07:24:09 on dist-test-slave-44d2"
server_key: "5d27df9fb1b87f68e50a8cd8b07ddeca"
server_key_iv: "fd6eae0f9ffdec44edd04a4a643ca0df"
server_key_version: "encryptionkey@0"
I20260810 07:24:09.371061 30233 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:24:09.373515 30256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:09.374716 30233 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:24:09.374852 30233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "fe921b351de843138c03547374f56080"
format_stamp: "Formatted at 2026-08-10 07:24:09 on dist-test-slave-44d2"
server_key: "5d27df9fb1b87f68e50a8cd8b07ddeca"
server_key_iv: "fd6eae0f9ffdec44edd04a4a643ca0df"
server_key_version: "encryptionkey@0"
I20260810 07:24:09.374959 30233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:09.383967 30233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:09.384675 30233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:09.384865 30233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:09.385434 30233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:09.386425 30233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:09.386474 30233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:09.386538 30233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:09.386571 30233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:09.396121 30233 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:35075
I20260810 07:24:09.396242 30425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:35075 every 8 connection(s)
I20260810 07:24:09.397135 30233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:24:09.406057 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.398929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:44535 (local address 127.1.125.254:46513)
0810 07:24:09.399118 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:09.399123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:09.400204 (+ 1081us) server_negotiation.cc:407] Connection header received
0810 07:24:09.401624 (+ 1420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:09.401627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:09.401676 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:09.401753 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:09.402625 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.403165 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.404258 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.404466 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.404609 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.405587 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:09.405590 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:09.405592 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:09.405605 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:09.405622 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:09.405630 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:09.405690 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:09.405851 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 07:24:09.405905 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 07:24:09.406277 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 30233
I20260810 07:24:09.406432 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:24:09.406827 1527 external_mini_cluster.cc:1428] Setting key 770df5b59b925542cf20a6f29a57f4e0
I20260810 07:24:09.406996 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.399373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:46513 (local address 127.1.125.194:44535)
0810 07:24:09.400042 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:09.400083 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:09.401414 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:09.401918 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:09.401924 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:09.402006 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:09.402480 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.402507 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.403271 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.403274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.404150 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.404159 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.404645 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.404648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.404785 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.405336 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:24:09.405337 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:09.405375 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:24:09.405437 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:09.405704 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:09.405711 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:09.405715 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:24:09.405825 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 07:24:09.406049 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":133,"threads_started":1}
I20260810 07:24:09.408315 30427 heartbeater.cc:347] Connected to a master server at 127.1.125.254:46513
I20260810 07:24:09.408584 30427 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:09.408936 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46513
--builtin_ntp_servers=127.1.125.212:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:24:09.409127 30427 heartbeater.cc:511] Master 127.1.125.254:46513 requested a full tablet report, sending...
I20260810 07:24:09.410120 29950 ts_manager.cc:194] Registered new tserver with Master: fe921b351de843138c03547374f56080 (127.1.125.194:35075)
I20260810 07:24:09.410660 29950 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.125.194:44535
W20260810 07:24:09.526943 30433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:09.527209 30433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:09.527247 30433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:09.531255 30433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:09.531315 30433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:09.531368 30433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:24:09.536036 30433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:46513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:09.537087 30433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:09.538450 30433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:09.548271 30446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:09.548317 30440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:09.548290 30444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:09.548348 30433 server_base.cc:1034] running on GCE node
I20260810 07:24:09.548969 30433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:09.549628 30433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:09.550832 30433 hybrid_clock.cc:648] HybridClock initialized: now 1786346649550817 us; error 29 us; skew 500 ppm
I20260810 07:24:09.552843 30433 webserver.cc:460] Webserver started at http://127.1.125.195:37967/ using document root <none> and password file <none>
I20260810 07:24:09.553483 30433 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:09.553583 30433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:09.553835 30433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:09.555979 30433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "8f62fffc8fa84fbabe3445ddd435f1e9"
format_stamp: "Formatted at 2026-08-10 07:24:09 on dist-test-slave-44d2"
server_key: "28c6b02ed5b092a0af3ddb75639e5103"
server_key_iv: "b038d2e07b52c70b28409a656eb0eb32"
server_key_version: "encryptionkey@0"
I20260810 07:24:09.556638 30433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "8f62fffc8fa84fbabe3445ddd435f1e9"
format_stamp: "Formatted at 2026-08-10 07:24:09 on dist-test-slave-44d2"
server_key: "28c6b02ed5b092a0af3ddb75639e5103"
server_key_iv: "b038d2e07b52c70b28409a656eb0eb32"
server_key_version: "encryptionkey@0"
I20260810 07:24:09.560282 30433 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:24:09.562748 30455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:09.563970 30433 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:24:09.564123 30433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "8f62fffc8fa84fbabe3445ddd435f1e9"
format_stamp: "Formatted at 2026-08-10 07:24:09 on dist-test-slave-44d2"
server_key: "28c6b02ed5b092a0af3ddb75639e5103"
server_key_iv: "b038d2e07b52c70b28409a656eb0eb32"
server_key_version: "encryptionkey@0"
I20260810 07:24:09.564244 30433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:09.588649 30433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:09.589468 30433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:09.589727 30433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:09.590399 30433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:09.591424 30433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:09.591506 30433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:09.591584 30433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:09.591629 30433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:09.601881 30433 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:46221
I20260810 07:24:09.601912 30623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:46221 every 8 connection(s)
I20260810 07:24:09.602842 30433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:24:09.606818 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 30433
I20260810 07:24:09.606961 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:24:09.607265 1527 external_mini_cluster.cc:1428] Setting key 02ec9a04ff9ab88a8517f15f49b47b29
I20260810 07:24:09.612742 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.604775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:51601 (local address 127.1.125.254:46513)
0810 07:24:09.604924 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:09.604927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:09.606037 (+ 1110us) server_negotiation.cc:407] Connection header received
0810 07:24:09.607637 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:09.607640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:09.607689 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:09.607764 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:09.608904 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.609548 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.610732 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.610957 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.611071 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.612194 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:09.612197 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:09.612200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:09.612221 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:09.612240 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:09.612249 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:09.612335 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:09.612518 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 07:24:09.612586 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:24:09.613605 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.605184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:46513 (local address 127.1.125.195:51601)
0810 07:24:09.605858 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:09.605900 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:09.607409 (+ 1509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:09.607976 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:09.607983 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:09.608068 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:09.608759 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.608776 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.609691 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.609694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.610602 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.610612 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.611115 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.611119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.611292 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.611887 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:24:09.611889 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:09.611929 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:24:09.612039 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:09.612338 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:09.612350 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:09.612358 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:24:09.612519 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 07:24:09.612834 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":108,"threads_started":1}
I20260810 07:24:09.614701 30624 heartbeater.cc:347] Connected to a master server at 127.1.125.254:46513
I20260810 07:24:09.614920 30624 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:09.615392 30624 heartbeater.cc:511] Master 127.1.125.254:46513 requested a full tablet report, sending...
I20260810 07:24:09.616380 29950 ts_manager.cc:194] Registered new tserver with Master: 8f62fffc8fa84fbabe3445ddd435f1e9 (127.1.125.195:46221)
I20260810 07:24:09.616969 29950 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.125.195:51601
I20260810 07:24:09.621260 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:24:09.626897 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.623156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39986 (local address 127.1.125.254:46513)
0810 07:24:09.623322 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:09.623327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:09.623499 (+ 172us) server_negotiation.cc:407] Connection header received
0810 07:24:09.623607 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:09.623609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:09.623650 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:09.623715 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:09.623920 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.624420 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.625279 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.625486 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.625567 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.626411 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:09.626414 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:09.626416 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:09.626427 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:09.626443 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:09.626450 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:09.626542 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:09.626672 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 07:24:09.626734 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 07:24:09.629906 29950 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:24:09.632953 29950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:24:09.646698 30639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.640970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:46469 (local address 127.0.0.1:60648)
0810 07:24:09.641457 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:09.641504 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:09.641631 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:09.642506 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:09.642514 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:09.642541 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:09.642701 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.642709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.643919 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.643922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.645027 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.645063 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.646163 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.646166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.646281 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.646323 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 07:24:09.646434 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 07:24:09.646556 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":73,"threads_started":1}
I20260810 07:24:09.646752 30642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.641145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60648 (local address 127.1.125.193:46469)
0810 07:24:09.641946 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:09.641968 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:09.642009 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:24:09.642094 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:09.642102 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:09.642326 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:09.642491 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:09.642856 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.643794 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.645206 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.646041 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.646160 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.646350 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:09.646475 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 07:24:09.646582 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":132,"threads_started":1}
I20260810 07:24:09.647486 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.641167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:35075 (local address 127.0.0.1:45556)
0810 07:24:09.641812 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:09.641825 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:09.641907 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:09.643247 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:09.643252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:09.643272 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:09.643370 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.643376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.644890 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.644895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.646003 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.646014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.647071 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.647074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.647194 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.647213 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:24:09.647280 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 07:24:09.647345 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":57,"thread_start_us":107,"threads_started":1}
I20260810 07:24:09.647559 30645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.641904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45556 (local address 127.1.125.194:35075)
0810 07:24:09.642804 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:09.642815 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:09.642836 (+ 21us) server_negotiation.cc:407] Connection header received
0810 07:24:09.642905 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:09.642909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:09.643095 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:09.643529 (+ 434us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:09.643679 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.644778 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.646137 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.646942 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.647058 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.647252 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:09.647308 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 07:24:09.647406 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":168,"threads_started":1}
I20260810 07:24:09.648177 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.641667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:46221 (local address 127.0.0.1:42832)
0810 07:24:09.642225 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:09.642238 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:09.642330 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:09.643101 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:09.643104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:09.643124 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:09.643209 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.643215 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.644584 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.644595 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.645767 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.645778 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.646957 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.646960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.647062 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.647769 (+ 707us) client_negotiation.cc:769] Sending connection context
0810 07:24:09.647835 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 07:24:09.648039 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":675,"thread_start_us":105,"threads_started":1}
I20260810 07:24:09.648231 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.642017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42832 (local address 127.1.125.195:46221)
0810 07:24:09.642612 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:09.642626 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:09.642652 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:24:09.642730 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:09.642735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:09.642919 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:09.643107 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:09.643309 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.644449 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.645916 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.646727 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.647156 (+ 429us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.647349 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:09.647962 (+ 613us) server_negotiation.cc:299] Negotiation successful
0810 07:24:09.648078 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":176,"threads_started":1}
I20260810 07:24:09.650220 30134 tablet_service.cc:1467] Processing CreateTablet for tablet af1eb8c5e5114d7fb8bc6b0543e22ad0 (DEFAULT_TABLE table=test-table [id=9ab0724f7892411092ee65f5f057e6d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:09.650492 30326 tablet_service.cc:1467] Processing CreateTablet for tablet af1eb8c5e5114d7fb8bc6b0543e22ad0 (DEFAULT_TABLE table=test-table [id=9ab0724f7892411092ee65f5f057e6d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:09.651310 30134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1eb8c5e5114d7fb8bc6b0543e22ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:09.651119 30524 tablet_service.cc:1467] Processing CreateTablet for tablet af1eb8c5e5114d7fb8bc6b0543e22ad0 (DEFAULT_TABLE table=test-table [id=9ab0724f7892411092ee65f5f057e6d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:09.651571 30326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1eb8c5e5114d7fb8bc6b0543e22ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:09.652096 30524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1eb8c5e5114d7fb8bc6b0543e22ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:09.658296 30649 tablet_bootstrap.cc:492] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6: Bootstrap starting.
I20260810 07:24:09.659071 30652 tablet_bootstrap.cc:492] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080: Bootstrap starting.
I20260810 07:24:09.659679 30651 tablet_bootstrap.cc:492] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9: Bootstrap starting.
I20260810 07:24:09.660727 30649 tablet_bootstrap.cc:654] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:09.661298 30652 tablet_bootstrap.cc:654] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:09.661535 30651 tablet_bootstrap.cc:654] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:09.661674 30649 log.cc:824] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:09.662176 30652 log.cc:824] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:09.662379 30651 log.cc:824] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:09.663968 30649 tablet_bootstrap.cc:492] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6: No bootstrap required, opened a new log
I20260810 07:24:09.664176 30649 ts_tablet_manager.cc:1397] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:24:09.664395 30652 tablet_bootstrap.cc:492] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080: No bootstrap required, opened a new log
I20260810 07:24:09.664625 30652 ts_tablet_manager.cc:1397] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:24:09.667611 30649 raft_consensus.cc:348] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46a55ecb4640888ae43be7702bbef6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 46469 } } peers { permanent_uuid: "fe921b351de843138c03547374f56080" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35075 } } peers { permanent_uuid: "8f62fffc8fa84fbabe3445ddd435f1e9" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 46221 } }
I20260810 07:24:09.667945 30649 raft_consensus.cc:374] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:09.667771 30652 raft_consensus.cc:348] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46a55ecb4640888ae43be7702bbef6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 46469 } } peers { permanent_uuid: "fe921b351de843138c03547374f56080" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35075 } } peers { permanent_uuid: "8f62fffc8fa84fbabe3445ddd435f1e9" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 46221 } }
I20260810 07:24:09.668040 30649 raft_consensus.cc:729] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c46a55ecb4640888ae43be7702bbef6, State: Initialized, Role: FOLLOWER
I20260810 07:24:09.668038 30652 raft_consensus.cc:374] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:09.668100 30652 raft_consensus.cc:729] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe921b351de843138c03547374f56080, State: Initialized, Role: FOLLOWER
I20260810 07:24:09.668612 30649 consensus_queue.cc:260] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46a55ecb4640888ae43be7702bbef6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 46469 } } peers { permanent_uuid: "fe921b351de843138c03547374f56080" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35075 } } peers { permanent_uuid: "8f62fffc8fa84fbabe3445ddd435f1e9" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 46221 } }
I20260810 07:24:09.668893 30651 tablet_bootstrap.cc:492] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9: No bootstrap required, opened a new log
I20260810 07:24:09.668829 30652 consensus_queue.cc:260] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46a55ecb4640888ae43be7702bbef6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 46469 } } peers { permanent_uuid: "fe921b351de843138c03547374f56080" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35075 } } peers { permanent_uuid: "8f62fffc8fa84fbabe3445ddd435f1e9" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 46221 } }
I20260810 07:24:09.669102 30651 ts_tablet_manager.cc:1397] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260810 07:24:09.669427 30229 heartbeater.cc:503] Master 127.1.125.254:46513 was elected leader, sending a full tablet report...
I20260810 07:24:09.669914 30649 ts_tablet_manager.cc:1428] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260810 07:24:09.670969 30652 ts_tablet_manager.cc:1428] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:24:09.671324 30427 heartbeater.cc:503] Master 127.1.125.254:46513 was elected leader, sending a full tablet report...
I20260810 07:24:09.674041 30651 raft_consensus.cc:348] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46a55ecb4640888ae43be7702bbef6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 46469 } } peers { permanent_uuid: "fe921b351de843138c03547374f56080" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35075 } } peers { permanent_uuid: "8f62fffc8fa84fbabe3445ddd435f1e9" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 46221 } }
I20260810 07:24:09.674342 30651 raft_consensus.cc:374] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:09.674446 30651 raft_consensus.cc:729] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f62fffc8fa84fbabe3445ddd435f1e9, State: Initialized, Role: FOLLOWER
I20260810 07:24:09.674935 30651 consensus_queue.cc:260] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46a55ecb4640888ae43be7702bbef6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 46469 } } peers { permanent_uuid: "fe921b351de843138c03547374f56080" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35075 } } peers { permanent_uuid: "8f62fffc8fa84fbabe3445ddd435f1e9" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 46221 } }
I20260810 07:24:09.675529 30624 heartbeater.cc:503] Master 127.1.125.254:46513 was elected leader, sending a full tablet report...
I20260810 07:24:09.675742 30651 ts_tablet_manager.cc:1428] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260810 07:24:09.709203 30231 tablet.cc:2369] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:24:09.853911 30625 tablet.cc:2369] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:24:09.898303 30430 tablet.cc:2369] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:24:09.941553 30659 raft_consensus.cc:482] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:24:09.941836 30659 raft_consensus.cc:504] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46a55ecb4640888ae43be7702bbef6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 46469 } } peers { permanent_uuid: "fe921b351de843138c03547374f56080" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35075 } } peers { permanent_uuid: "8f62fffc8fa84fbabe3445ddd435f1e9" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 46221 } }
I20260810 07:24:09.943214 30659 leader_election.cc:288] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c46a55ecb4640888ae43be7702bbef6 (127.1.125.193:46469), 8f62fffc8fa84fbabe3445ddd435f1e9 (127.1.125.195:46221)
I20260810 07:24:09.948235 30642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.943631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:38041 (local address 127.1.125.193:46469)
0810 07:24:09.943828 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:09.943834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:09.943935 (+ 101us) server_negotiation.cc:407] Connection header received
0810 07:24:09.944262 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:09.944265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:09.944314 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:09.944408 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:09.944780 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.945577 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.947134 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.947523 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.947671 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.947724 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:09.947966 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 07:24:09.948062 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 07:24:09.948432 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.943487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:46221 (local address 127.1.125.194:54327)
0810 07:24:09.943934 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:09.943948 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:09.944051 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:09.944565 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:09.944570 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:09.944598 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:09.944730 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.944740 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.945527 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.945529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.946953 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.946966 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.947968 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.947973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.948085 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.948108 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:24:09.948186 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:24:09.948262 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":107,"spinlock_wait_cycles":25728,"thread_start_us":126,"threads_started":1}
I20260810 07:24:09.948443 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.944106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:54327 (local address 127.1.125.195:46221)
0810 07:24:09.944247 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:09.944252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:09.944273 (+ 21us) server_negotiation.cc:407] Connection header received
0810 07:24:09.944342 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:09.944346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:09.944402 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:09.944538 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:09.944871 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.945396 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.947113 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.947779 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.947981 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.948052 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:09.948222 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 07:24:09.948302 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:24:09.948709 30662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:09.943487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:46469 (local address 127.1.125.194:38041)
0810 07:24:09.943838 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:09.943855 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:09.944051 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:09.944427 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:09.944431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:09.944452 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:09.944618 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.944625 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.945711 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.945714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.946953 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:09.946966 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:09.947665 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:09.947669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:09.947794 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:09.947815 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:24:09.948450 (+ 635us) client_negotiation.cc:240] Negotiation successful
0810 07:24:09.948571 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":59,"spinlock_wait_cycles":15616,"thread_start_us":121,"threads_started":1}
I20260810 07:24:09.949206 30551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1eb8c5e5114d7fb8bc6b0543e22ad0" candidate_uuid: "fe921b351de843138c03547374f56080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f62fffc8fa84fbabe3445ddd435f1e9" is_pre_election: true
I20260810 07:24:09.949568 30551 raft_consensus.cc:2454] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe921b351de843138c03547374f56080 in term 0.
I20260810 07:24:09.949577 30164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1eb8c5e5114d7fb8bc6b0543e22ad0" candidate_uuid: "fe921b351de843138c03547374f56080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c46a55ecb4640888ae43be7702bbef6" is_pre_election: true
I20260810 07:24:09.949909 30164 raft_consensus.cc:2454] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe921b351de843138c03547374f56080 in term 0.
I20260810 07:24:09.950165 30260 leader_election.cc:302] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f62fffc8fa84fbabe3445ddd435f1e9, fe921b351de843138c03547374f56080; no voters:
I20260810 07:24:09.950516 30659 raft_consensus.cc:2781] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:24:09.950632 30659 raft_consensus.cc:482] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:24:09.950693 30659 raft_consensus.cc:3037] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:09.951793 30659 raft_consensus.cc:504] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46a55ecb4640888ae43be7702bbef6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 46469 } } peers { permanent_uuid: "fe921b351de843138c03547374f56080" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35075 } } peers { permanent_uuid: "8f62fffc8fa84fbabe3445ddd435f1e9" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 46221 } }
I20260810 07:24:09.952214 30659 leader_election.cc:288] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [CANDIDATE]: Term 1 election: Requested vote from peers 6c46a55ecb4640888ae43be7702bbef6 (127.1.125.193:46469), 8f62fffc8fa84fbabe3445ddd435f1e9 (127.1.125.195:46221)
I20260810 07:24:09.952574 30164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1eb8c5e5114d7fb8bc6b0543e22ad0" candidate_uuid: "fe921b351de843138c03547374f56080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c46a55ecb4640888ae43be7702bbef6"
I20260810 07:24:09.952689 30551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1eb8c5e5114d7fb8bc6b0543e22ad0" candidate_uuid: "fe921b351de843138c03547374f56080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f62fffc8fa84fbabe3445ddd435f1e9"
I20260810 07:24:09.952733 30164 raft_consensus.cc:3037] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:09.952822 30551 raft_consensus.cc:3037] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:09.954115 30551 raft_consensus.cc:2454] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe921b351de843138c03547374f56080 in term 1.
I20260810 07:24:09.954110 30164 raft_consensus.cc:2454] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe921b351de843138c03547374f56080 in term 1.
I20260810 07:24:09.954480 30257 leader_election.cc:302] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c46a55ecb4640888ae43be7702bbef6, fe921b351de843138c03547374f56080; no voters:
I20260810 07:24:09.954737 30659 raft_consensus.cc:2781] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:09.954986 30659 raft_consensus.cc:686] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [term 1 LEADER]: Becoming Leader. State: Replica: fe921b351de843138c03547374f56080, State: Running, Role: LEADER
I20260810 07:24:09.955416 30659 consensus_queue.cc:237] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46a55ecb4640888ae43be7702bbef6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 46469 } } peers { permanent_uuid: "fe921b351de843138c03547374f56080" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35075 } } peers { permanent_uuid: "8f62fffc8fa84fbabe3445ddd435f1e9" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 46221 } }
I20260810 07:24:09.958775 29950 catalog_manager.cc:5495] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe921b351de843138c03547374f56080 (127.1.125.194). New cstate: current_term: 1 leader_uuid: "fe921b351de843138c03547374f56080" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46a55ecb4640888ae43be7702bbef6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 46469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe921b351de843138c03547374f56080" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 35075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f62fffc8fa84fbabe3445ddd435f1e9" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 46221 } health_report { overall_health: UNKNOWN } } }
I20260810 07:24:10.011062 30645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:10.007252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45566 (local address 127.1.125.194:35075)
0810 07:24:10.007399 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:10.007404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:10.007421 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:24:10.007571 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:10.007574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:10.007634 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:10.007759 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:10.008043 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:10.008567 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:10.009456 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:10.009682 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:10.009802 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:10.010138 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:10.010542 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:10.010724 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:10.010857 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 07:24:10.010892 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 07:24:10.017041 30551 raft_consensus.cc:1261] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9 [term 1 FOLLOWER]: Refusing update from remote peer fe921b351de843138c03547374f56080: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:10.017042 30164 raft_consensus.cc:1261] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 6c46a55ecb4640888ae43be7702bbef6 [term 1 FOLLOWER]: Refusing update from remote peer fe921b351de843138c03547374f56080: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:10.017788 30659 consensus_queue.cc:1035] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c46a55ecb4640888ae43be7702bbef6" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 46469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:10.018033 30664 consensus_queue.cc:1035] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f62fffc8fa84fbabe3445ddd435f1e9" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 46221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:10.026957 30668 mvcc.cc:205] Tried to move back new op lower bound from 7316875878462558208 to 7316875878220677120. Current Snapshot: MvccSnapshot[applied={T|T < 7316875878462558208}]
I20260810 07:24:10.028934 30669 mvcc.cc:205] Tried to move back new op lower bound from 7316875878462558208 to 7316875878220677120. Current Snapshot: MvccSnapshot[applied={T|T < 7316875878462558208 or (T in {7316875878462558208})}]
I20260810 07:24:10.033820 29949 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:24:10.033964 29949 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:24:10.036267 29949 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e0d8f1bbfa7458caba14214da2db2d4: Sending DeleteTablet for 3 replicas of tablet af1eb8c5e5114d7fb8bc6b0543e22ad0
I20260810 07:24:10.037017 30134 tablet_service.cc:1514] Processing DeleteTablet for tablet af1eb8c5e5114d7fb8bc6b0543e22ad0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:10 UTC) from {username='slave'} at 127.0.0.1:60648
I20260810 07:24:10.037312 30326 tablet_service.cc:1514] Processing DeleteTablet for tablet af1eb8c5e5114d7fb8bc6b0543e22ad0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:10 UTC) from {username='slave'} at 127.0.0.1:45556
I20260810 07:24:10.037761 30524 tablet_service.cc:1514] Processing DeleteTablet for tablet af1eb8c5e5114d7fb8bc6b0543e22ad0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:10 UTC) from {username='slave'} at 127.0.0.1:42832
I20260810 07:24:10.037850 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 30043
I20260810 07:24:10.049093 30681 tablet_replica.cc:330] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080: stopping tablet replica
I20260810 07:24:10.049621 30681 raft_consensus.cc:2229] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:24:10.050144 30681 raft_consensus.cc:2258] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:24:10.050171 30682 tablet_replica.cc:330] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9: stopping tablet replica
I20260810 07:24:10.050570 30682 raft_consensus.cc:2229] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:24:10.051012 30682 raft_consensus.cc:2258] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:24:10.051853 30681 ts_tablet_manager.cc:1905] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:24:10.052443 30682 ts_tablet_manager.cc:1905] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:24:10.055225 30682 ts_tablet_manager.cc:1918] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:24:10.055433 30682 log.cc:1196] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/af1eb8c5e5114d7fb8bc6b0543e22ad0
I20260810 07:24:10.055595 30681 ts_tablet_manager.cc:1918] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:24:10.055716 30681 log.cc:1196] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/af1eb8c5e5114d7fb8bc6b0543e22ad0
I20260810 07:24:10.055963 30682 ts_tablet_manager.cc:1939] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P 8f62fffc8fa84fbabe3445ddd435f1e9: Deleting consensus metadata
I20260810 07:24:10.056106 30681 ts_tablet_manager.cc:1939] T af1eb8c5e5114d7fb8bc6b0543e22ad0 P fe921b351de843138c03547374f56080: Deleting consensus metadata
I20260810 07:24:10.057370 29928 catalog_manager.cc:4841] TS 8f62fffc8fa84fbabe3445ddd435f1e9 (127.1.125.195:46221): tablet af1eb8c5e5114d7fb8bc6b0543e22ad0 (table test-table [id=9ab0724f7892411092ee65f5f057e6d5]) successfully deleted
I20260810 07:24:10.057629 29928 catalog_manager.cc:4841] TS fe921b351de843138c03547374f56080 (127.1.125.194:35075): tablet af1eb8c5e5114d7fb8bc6b0543e22ad0 (table test-table [id=9ab0724f7892411092ee65f5f057e6d5]) successfully deleted
W20260810 07:24:10.058789 29927 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:46469 (error 108)
I20260810 07:24:10.059706 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 30233
I20260810 07:24:10.059934 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:10.059617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:46469 (local address 127.0.0.1:60664)
0810 07:24:10.059754 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:10.059834 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:46469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 07:24:10.064316 29927 catalog_manager.cc:4584] TS 6c46a55ecb4640888ae43be7702bbef6 (127.1.125.193:46469): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet af1eb8c5e5114d7fb8bc6b0543e22ad0: Network error: Client connection negotiation failed: client connection to 127.1.125.193:46469: connect: Connection refused (error 111)
I20260810 07:24:10.070432 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 30433
I20260810 07:24:10.084801 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 29904
2026-08-10T07:24:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1356 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:10 dist-test-slave-44d2 krb5kdc[30692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:10 dist-test-slave-44d2 krb5kdc[30692](info): set up 2 sockets
Aug 10 07:24:10 dist-test-slave-44d2 krb5kdc[30692](info): commencing operation
krb5kdc: starting...
W20260810 07:24:11.445938 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.333s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:11 dist-test-slave-44d2 krb5kdc[30692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:24:11.579852 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:41323
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:41323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:11.695842 30708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:11.696118 30708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:11.696158 30708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:11.700170 30708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:11.700234 30708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:11.700253 30708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:11.700270 30708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:11.700286 30708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:11.705649 30708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:41323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:41323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:11.706761 30708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:11.707847 30708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:11.716739 30721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:11.716941 30713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:11.716749 30716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:11.717136 30708 server_base.cc:1034] running on GCE node
I20260810 07:24:11.717767 30708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:11.718950 30708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:11.720146 30708 hybrid_clock.cc:648] HybridClock initialized: now 1786346651720124 us; error 34 us; skew 500 ppm
Aug 10 07:24:11 dist-test-slave-44d2 krb5kdc[30692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346651, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:11.723081 30708 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:24:11.723971 30708 webserver.cc:460] Webserver started at http://127.1.125.254:41383/ using document root <none> and password file <none>
I20260810 07:24:11.724622 30708 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:11.724720 30708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:11.724967 30708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:11.727094 30708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "f6e4e25502884dda816a070945d0557b"
format_stamp: "Formatted at 2026-08-10 07:24:11 on dist-test-slave-44d2"
server_key: "441fc7491fa8236b3e7437c76e2bc4d0"
server_key_iv: "8e0ba33c57b9b55d6ed8f1cff45dc569"
server_key_version: "encryptionkey@0"
I20260810 07:24:11.727677 30708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "f6e4e25502884dda816a070945d0557b"
format_stamp: "Formatted at 2026-08-10 07:24:11 on dist-test-slave-44d2"
server_key: "441fc7491fa8236b3e7437c76e2bc4d0"
server_key_iv: "8e0ba33c57b9b55d6ed8f1cff45dc569"
server_key_version: "encryptionkey@0"
I20260810 07:24:11.731594 30708 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:24:11.734512 30729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:11.735678 30708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:24:11.735812 30708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "f6e4e25502884dda816a070945d0557b"
format_stamp: "Formatted at 2026-08-10 07:24:11 on dist-test-slave-44d2"
server_key: "441fc7491fa8236b3e7437c76e2bc4d0"
server_key_iv: "8e0ba33c57b9b55d6ed8f1cff45dc569"
server_key_version: "encryptionkey@0"
I20260810 07:24:11.735921 30708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:11.747784 30708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:11.750571 30708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:11.750785 30708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:11.758301 30708 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:41323
I20260810 07:24:11.758328 30809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:41323 every 8 connection(s)
I20260810 07:24:11.759351 30708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:11.762341 30810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:11.767719 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 30708
I20260810 07:24:11.767836 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:11.768153 1527 external_mini_cluster.cc:1428] Setting key 6e35ed6335820941145e1ded4401eefa
I20260810 07:24:11.768151 30810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b: Bootstrap starting.
I20260810 07:24:11.770754 30810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:11.771514 30810 log.cc:824] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:11.773569 30810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b: No bootstrap required, opened a new log
Aug 10 07:24:11 dist-test-slave-44d2 krb5kdc[30692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:24:11.777150 30810 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e4e25502884dda816a070945d0557b" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 41323 } }
I20260810 07:24:11.777396 30810 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:11.777463 30810 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e4e25502884dda816a070945d0557b, State: Initialized, Role: FOLLOWER
I20260810 07:24:11.777985 30810 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e4e25502884dda816a070945d0557b" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 41323 } }
I20260810 07:24:11.778190 30810 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:11.778272 30810 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:11.778390 30810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:11.779603 30810 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e4e25502884dda816a070945d0557b" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 41323 } }
I20260810 07:24:11.779994 30810 leader_election.cc:302] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6e4e25502884dda816a070945d0557b; no voters:
I20260810 07:24:11.780347 30810 leader_election.cc:288] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:11.780438 30819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:11.780737 30819 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [term 1 LEADER]: Becoming Leader. State: Replica: f6e4e25502884dda816a070945d0557b, State: Running, Role: LEADER
I20260810 07:24:11.781068 30819 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e4e25502884dda816a070945d0557b" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 41323 } }
I20260810 07:24:11.781720 30810 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:11.782300 30821 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6e4e25502884dda816a070945d0557b. Latest consensus state: current_term: 1 leader_uuid: "f6e4e25502884dda816a070945d0557b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e4e25502884dda816a070945d0557b" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 41323 } } }
I20260810 07:24:11.782518 30821 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:11.782794 30822 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6e4e25502884dda816a070945d0557b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e4e25502884dda816a070945d0557b" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 41323 } } }
I20260810 07:24:11.782893 30822 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:11.783118 30831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:11.783818 30814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:11.769612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39874 (local address 127.1.125.254:41323)
0810 07:24:11.770055 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:11.770070 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:11.770106 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:24:11.770714 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:11.770730 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:11.771118 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:11.771454 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:11.772302 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:11.773350 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:11.774295 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:11.774765 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:11.774953 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:11.777559 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:11.777565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:11.777582 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:11.777625 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:11.779792 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:11.780289 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:11.780298 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:11.780306 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:11.780459 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:11.780910 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:11.780914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:11.780916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:11.781451 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:11.781656 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:11.782347 (+ 691us) server_negotiation.cc:299] Negotiation successful
0810 07:24:11.782712 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":137,"threads_started":1}
I20260810 07:24:11.786652 30831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:11.792747 30831 catalog_manager.cc:1347] Generated new cluster ID: 177d406edfce4bf39b135c681e78c98e
I20260810 07:24:11.792814 30831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:11.802030 30831 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:11.803227 30831 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:11.812064 30831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b: Generated new TSK 0
I20260810 07:24:11.812925 30831 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
Entry for principal kudu/127.1.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab.
I20260810 07:24:11.844903 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:41323
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:11.967880 30848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:11.968151 30848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:11.968247 30848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:11.974809 30848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:11.974915 30848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:11.975055 30848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:24:11.982306 30848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.193.keytab
--principal=kudu/127.1.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:41323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:11.983639 30848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:11.985140 30848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:12.000020 30859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:12.000015 30862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:12.000334 30848 server_base.cc:1034] running on GCE node
W20260810 07:24:12.000352 30865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:12.000936 30848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:12.001654 30848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:12.002852 30848 hybrid_clock.cc:648] HybridClock initialized: now 1786346652002815 us; error 33 us; skew 500 ppm
Aug 10 07:24:12 dist-test-slave-44d2 krb5kdc[30692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346652, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:12.006516 30848 init.cc:377] Logged in from keytab as kudu/127.1.125.193@KRBTEST.COM (short username kudu)
I20260810 07:24:12.009887 30848 webserver.cc:460] Webserver started at http://127.1.125.193:35587/ using document root <none> and password file <none>
I20260810 07:24:12.010723 30848 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:12.010816 30848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:12.011058 30848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:12.013433 30848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "45c403e832f44b5d9838eea2204e487c"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-44d2"
server_key: "372b12db5315b2ada3805b2632c4361e"
server_key_iv: "f94da3981fb0fad3542eee56aee414c0"
server_key_version: "encryptionkey@0"
I20260810 07:24:12.014120 30848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "45c403e832f44b5d9838eea2204e487c"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-44d2"
server_key: "372b12db5315b2ada3805b2632c4361e"
server_key_iv: "f94da3981fb0fad3542eee56aee414c0"
server_key_version: "encryptionkey@0"
I20260810 07:24:12.019400 30848 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 07:24:12.022792 30875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:12.024420 30848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:24:12.024577 30848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "45c403e832f44b5d9838eea2204e487c"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-44d2"
server_key: "372b12db5315b2ada3805b2632c4361e"
server_key_iv: "f94da3981fb0fad3542eee56aee414c0"
server_key_version: "encryptionkey@0"
I20260810 07:24:12.024693 30848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:12.056584 30848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:12.059441 30848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:12.059657 30848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:12.060303 30848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:12.061182 30848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:12.061229 30848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:12.061302 30848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:12.061336 30848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:12.070465 30848 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:33381
I20260810 07:24:12.070493 31044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:33381 every 8 connection(s)
I20260810 07:24:12.071364 30848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:24:12.074124 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 30848
I20260810 07:24:12.074290 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:24:12.074697 1527 external_mini_cluster.cc:1428] Setting key 1d0138f1793f988789aa710c18ee1c34
Aug 10 07:24:12 dist-test-slave-44d2 krb5kdc[30692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346652, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.193@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:24:12.086833 30814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.073316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:51633 (local address 127.1.125.254:41323)
0810 07:24:12.073487 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:12.073492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:12.074586 (+ 1094us) server_negotiation.cc:407] Connection header received
0810 07:24:12.075669 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:12.075672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:12.075726 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:12.075807 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:12.077024 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.077565 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.079165 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.079404 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.079491 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.082676 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:12.082682 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:12.082689 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:12.082725 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:12.084657 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:12.085175 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:12.085179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:12.085182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:12.085288 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:12.085634 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:12.085637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:12.085639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:12.085903 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:12.086015 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:12.086510 (+ 495us) server_negotiation.cc:299] Negotiation successful
0810 07:24:12.086659 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
WARNING: no policy specified for kudu/127.1.125.194@KRBTEST.COM; defaulting to no policy
I20260810 07:24:12.087627 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.073779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:41323 (local address 127.1.125.193:51633)
0810 07:24:12.074304 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:12.074437 (+ 133us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:12.075456 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:12.075981 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:12.075988 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:12.076359 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:12.076873 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.076888 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.077727 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.077730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.079024 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.079039 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.079574 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.079579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.079817 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.080546 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:12.080562 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:12.082481 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:12.084775 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:12.084780 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:12.084792 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:12.085056 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:12.085396 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:12.085399 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:12.085400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:12.085536 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:12.086009 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:12.086013 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:12.086253 (+ 240us) client_negotiation.cc:769] Sending connection context
0810 07:24:12.086410 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 07:24:12.086668 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":132,"threads_started":1}
I20260810 07:24:12.089097 31045 heartbeater.cc:347] Connected to a master server at 127.1.125.254:41323
I20260810 07:24:12.089350 31045 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:12.089860 31045 heartbeater.cc:511] Master 127.1.125.254:41323 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.194@KRBTEST.COM" created.
I20260810 07:24:12.091529 30751 ts_manager.cc:194] Registered new tserver with Master: 45c403e832f44b5d9838eea2204e487c (127.1.125.193:33381)
I20260810 07:24:12.093325 30751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.193@KRBTEST.COM'} at 127.1.125.193:51633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
Entry for principal kudu/127.1.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab.
I20260810 07:24:12.108805 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:41323
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:12.226693 31051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:12.226972 31051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:12.227020 31051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:12.231310 31051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:12.231364 31051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:12.231427 31051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:24:12.236617 31051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.194.keytab
--principal=kudu/127.1.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:41323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:12.237572 31051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:12.238551 31051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:12.248647 31059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:12.248627 31065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:12.248816 31051 server_base.cc:1034] running on GCE node
W20260810 07:24:12.248627 31060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:12.249334 31051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:12.250038 31051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:12.251226 31051 hybrid_clock.cc:648] HybridClock initialized: now 1786346652251211 us; error 26 us; skew 500 ppm
Aug 10 07:24:12 dist-test-slave-44d2 krb5kdc[30692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346652, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:12.254068 31051 init.cc:377] Logged in from keytab as kudu/127.1.125.194@KRBTEST.COM (short username kudu)
I20260810 07:24:12.255057 31051 webserver.cc:460] Webserver started at http://127.1.125.194:40597/ using document root <none> and password file <none>
I20260810 07:24:12.255722 31051 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:12.255816 31051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:12.256068 31051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:12.258322 31051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "cf83e53225d447b098c47a9cc4b84a2e"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-44d2"
server_key: "1e6f340963eb66c0fcb50838fefea92d"
server_key_iv: "9c10439adc8d35f16ad99be72131f3f3"
server_key_version: "encryptionkey@0"
I20260810 07:24:12.258950 31051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "cf83e53225d447b098c47a9cc4b84a2e"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-44d2"
server_key: "1e6f340963eb66c0fcb50838fefea92d"
server_key_iv: "9c10439adc8d35f16ad99be72131f3f3"
server_key_version: "encryptionkey@0"
I20260810 07:24:12.262812 31051 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 07:24:12.265394 31074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:12.266651 31051 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:24:12.266795 31051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "cf83e53225d447b098c47a9cc4b84a2e"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-44d2"
server_key: "1e6f340963eb66c0fcb50838fefea92d"
server_key_iv: "9c10439adc8d35f16ad99be72131f3f3"
server_key_version: "encryptionkey@0"
I20260810 07:24:12.266916 31051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:12.278609 31051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:12.281543 31051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:12.281744 31051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:12.282363 31051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:12.283356 31051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:12.283407 31051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:12.283473 31051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:12.283510 31051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:12.293031 31051 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:46189
I20260810 07:24:12.293031 31256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:46189 every 8 connection(s)
I20260810 07:24:12.293998 31051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:24:12.296372 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 31051
I20260810 07:24:12.296535 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:24:12.296906 1527 external_mini_cluster.cc:1428] Setting key 34451e2349c14cead69f2212d4d48307
Aug 10 07:24:12 dist-test-slave-44d2 krb5kdc[30692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346652, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.194@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.195@KRBTEST.COM; defaulting to no policy
I20260810 07:24:12.309103 30814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.296122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:49555 (local address 127.1.125.254:41323)
0810 07:24:12.296270 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:12.296274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:12.297210 (+ 936us) server_negotiation.cc:407] Connection header received
0810 07:24:12.298254 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:12.298257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:12.298315 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:12.298392 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:12.299603 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.300154 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.301945 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.302168 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.302255 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.305326 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:12.305330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:12.305333 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:12.305365 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:12.306851 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:12.307392 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:12.307400 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:12.307401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:12.307475 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:12.307843 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:12.307847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:12.307848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:12.308068 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:12.308183 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:12.308751 (+ 568us) server_negotiation.cc:299] Negotiation successful
0810 07:24:12.308919 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:24:12.309927 31259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.296420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:41323 (local address 127.1.125.194:49555)
0810 07:24:12.297025 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:12.297084 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:12.298022 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:12.298569 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:12.298576 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:12.298957 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:12.299441 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.299454 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.300289 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.300292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.301805 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.301820 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.302336 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.302343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.302595 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.303251 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:12.303278 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:12.305154 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:12.306997 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:12.307002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:12.307017 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:12.307277 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:12.307589 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:12.307592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:12.307594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:12.307738 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:12.308193 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:12.308202 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:12.308442 (+ 240us) client_negotiation.cc:769] Sending connection context
0810 07:24:12.308693 (+ 251us) client_negotiation.cc:240] Negotiation successful
0810 07:24:12.309047 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":154,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.195@KRBTEST.COM" created.
I20260810 07:24:12.311336 31257 heartbeater.cc:347] Connected to a master server at 127.1.125.254:41323
I20260810 07:24:12.311632 31257 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:12.312270 31257 heartbeater.cc:511] Master 127.1.125.254:41323 requested a full tablet report, sending...
I20260810 07:24:12.313508 30751 ts_manager.cc:194] Registered new tserver with Master: cf83e53225d447b098c47a9cc4b84a2e (127.1.125.194:46189)
I20260810 07:24:12.314141 30751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.194@KRBTEST.COM'} at 127.1.125.194:49555
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
Entry for principal kudu/127.1.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab.
I20260810 07:24:12.327504 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:41323
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:12.444419 31262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:12.444789 31262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:12.444854 31262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:12.448884 31262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:12.448935 31262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:12.448993 31262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:24:12.454150 31262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.195.keytab
--principal=kudu/127.1.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:41323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:12.455099 31262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:12.456135 31262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:12.466403 31277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:12.466393 31271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:12.466393 31273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:12.466753 31262 server_base.cc:1034] running on GCE node
I20260810 07:24:12.467326 31262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:12.468102 31262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:12.469326 31262 hybrid_clock.cc:648] HybridClock initialized: now 1786346652469306 us; error 40 us; skew 500 ppm
Aug 10 07:24:12 dist-test-slave-44d2 krb5kdc[30692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346652, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:12.472760 31262 init.cc:377] Logged in from keytab as kudu/127.1.125.195@KRBTEST.COM (short username kudu)
I20260810 07:24:12.474148 31262 webserver.cc:460] Webserver started at http://127.1.125.195:39615/ using document root <none> and password file <none>
I20260810 07:24:12.475309 31262 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:12.475430 31262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:12.475680 31262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:12.477959 31262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "ca8b4a56225744e4bbb66fc51b95e74f"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-44d2"
server_key: "5cc8d527762cfe2a7871939211ff32e4"
server_key_iv: "753a14fb4e89bc7a3202f65cfe2ab6ec"
server_key_version: "encryptionkey@0"
I20260810 07:24:12.478538 31262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "ca8b4a56225744e4bbb66fc51b95e74f"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-44d2"
server_key: "5cc8d527762cfe2a7871939211ff32e4"
server_key_iv: "753a14fb4e89bc7a3202f65cfe2ab6ec"
server_key_version: "encryptionkey@0"
I20260810 07:24:12.482623 31262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:24:12.485396 31289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:12.486702 31262 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:24:12.486850 31262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "ca8b4a56225744e4bbb66fc51b95e74f"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-44d2"
server_key: "5cc8d527762cfe2a7871939211ff32e4"
server_key_iv: "753a14fb4e89bc7a3202f65cfe2ab6ec"
server_key_version: "encryptionkey@0"
I20260810 07:24:12.486950 31262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:12.518723 31262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:12.522496 31262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:12.522746 31262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:12.523445 31262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:12.524417 31262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:12.524468 31262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:12.524590 31262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:12.524627 31262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:12.534374 31262 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:36005
I20260810 07:24:12.534443 31450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:36005 every 8 connection(s)
I20260810 07:24:12.535480 31262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:24:12.544917 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 31262
I20260810 07:24:12.545101 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
Aug 10 07:24:12 dist-test-slave-44d2 krb5kdc[30692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346652, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.195@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:24:12.545467 1527 external_mini_cluster.cc:1428] Setting key 76e2ff0d5c06d400525bb9b83bd518ce
I20260810 07:24:12.549650 30814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.537466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:33775 (local address 127.1.125.254:41323)
0810 07:24:12.537633 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:12.537637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:12.538533 (+ 896us) server_negotiation.cc:407] Connection header received
0810 07:24:12.539813 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:12.539816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:12.539865 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:12.539940 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:12.541261 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.541852 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.543006 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.543233 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.543320 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.546125 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:12.546129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:12.546132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:12.546161 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:12.547766 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:12.548260 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:12.548263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:12.548264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:12.548340 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:12.548693 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:12.548695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:12.548696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:12.548871 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:12.548949 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:12.549409 (+ 460us) server_negotiation.cc:299] Negotiation successful
0810 07:24:12.549531 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 07:24:12.550434 31453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.537887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:41323 (local address 127.1.125.195:33775)
0810 07:24:12.538367 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:12.538413 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:12.539616 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:12.540124 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:12.540130 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:12.540580 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:12.541098 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.541113 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.541979 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.541982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.542884 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.542894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.543371 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.543374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.543514 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.544129 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:12.544146 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:12.545958 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:12.547899 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:12.547904 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:12.547916 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:12.548175 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:12.548401 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:12.548403 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:12.548404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:12.548607 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:12.548963 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:12.548967 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:12.549196 (+ 229us) client_negotiation.cc:769] Sending connection context
0810 07:24:12.549345 (+ 149us) client_negotiation.cc:240] Negotiation successful
0810 07:24:12.549600 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":121,"threads_started":1}
I20260810 07:24:12.551498 31451 heartbeater.cc:347] Connected to a master server at 127.1.125.254:41323
I20260810 07:24:12.551744 31451 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:12.552230 31451 heartbeater.cc:511] Master 127.1.125.254:41323 requested a full tablet report, sending...
I20260810 07:24:12.553314 30751 ts_manager.cc:194] Registered new tserver with Master: ca8b4a56225744e4bbb66fc51b95e74f (127.1.125.195:36005)
I20260810 07:24:12.553855 30751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.125.195@KRBTEST.COM'} at 127.1.125.195:33775
I20260810 07:24:12.559512 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:24:12.568851 30814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.561605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39890 (local address 127.1.125.254:41323)
0810 07:24:12.561733 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:12.561736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:12.561939 (+ 203us) server_negotiation.cc:407] Connection header received
0810 07:24:12.562044 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:12.562046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:12.562092 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:12.562157 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:12.562826 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.563326 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.564255 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.564462 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.564618 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.565877 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:12.565880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:12.565882 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:12.565904 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:12.567370 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:12.567744 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:12.567746 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:12.567748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:12.567822 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:12.568060 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:12.568062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:12.568063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:12.568276 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:12.568379 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:12.568643 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 07:24:12.568735 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:24:12.571705 30751 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:24:12.574810 30751 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:24:12.588974 31471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.582747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:46189 (local address 127.0.0.1:46594)
0810 07:24:12.583629 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:12.583677 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:12.583818 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:12.584926 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:12.584928 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:12.584943 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:12.585050 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.585055 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.586052 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.586058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.587287 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.587298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.588370 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.588374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.588519 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.588564 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 07:24:12.588682 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 07:24:12.588806 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":112,"threads_started":1}
I20260810 07:24:12.589030 31478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.583501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46594 (local address 127.1.125.194:46189)
0810 07:24:12.584246 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:12.584258 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:12.584283 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:24:12.584361 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:12.584368 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:12.584595 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:12.584746 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:12.585170 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.585899 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.587413 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.588235 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.588354 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.588553 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:12.588741 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 07:24:12.588845 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":126,"threads_started":1}
I20260810 07:24:12.589272 31475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.583093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:33381 (local address 127.0.0.1:41340)
0810 07:24:12.583939 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:12.583954 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:12.584062 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:12.584815 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:12.584818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:12.584835 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:12.584922 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.584927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.586266 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.586269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.587871 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.587880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.588906 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.588910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.589022 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.589054 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 07:24:12.589108 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 07:24:12.589175 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":48,"thread_start_us":81,"threads_started":1}
I20260810 07:24:12.589812 31473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.582788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52330 (local address 127.1.125.195:36005)
0810 07:24:12.583275 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:12.583288 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:12.583807 (+ 519us) server_negotiation.cc:407] Connection header received
0810 07:24:12.584073 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:12.584080 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:12.584302 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:12.584570 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:12.584869 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.586122 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.587297 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.588383 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.588507 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.588674 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:12.589558 (+ 884us) server_negotiation.cc:299] Negotiation successful
0810 07:24:12.589674 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":129,"threads_started":1}
I20260810 07:24:12.590034 31476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.583489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41340 (local address 127.1.125.193:33381)
0810 07:24:12.584311 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:12.584328 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:12.584354 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:24:12.584436 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:12.584443 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:12.584673 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:12.584843 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:12.585009 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.586130 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.588000 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.588778 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.589564 (+ 786us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.589728 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:12.589783 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 07:24:12.589880 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":109,"threads_started":1}
I20260810 07:24:12.590078 31469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.582627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:36005 (local address 127.0.0.1:52330)
0810 07:24:12.583618 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:12.583673 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:12.583875 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:12.584555 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:12.584561 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:12.584586 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:12.584720 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.584731 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.586319 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.586322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.587162 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.587171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.588745 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.588751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.588839 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.588861 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:24:12.588937 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:24:12.589002 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":20,"spinlock_wait_cycles":185088,"thread_start_us":106,"threads_started":1}
I20260810 07:24:12.592409 31156 tablet_service.cc:1467] Processing CreateTablet for tablet 72d8c86be6fa46618a6c1d22da5dcef7 (DEFAULT_TABLE table=test-table [id=76dc0574ce00478f934beed0dc7f62aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:12.592777 30946 tablet_service.cc:1467] Processing CreateTablet for tablet 72d8c86be6fa46618a6c1d22da5dcef7 (DEFAULT_TABLE table=test-table [id=76dc0574ce00478f934beed0dc7f62aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:12.593564 31156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d8c86be6fa46618a6c1d22da5dcef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:12.593868 30946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d8c86be6fa46618a6c1d22da5dcef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:12.593580 31357 tablet_service.cc:1467] Processing CreateTablet for tablet 72d8c86be6fa46618a6c1d22da5dcef7 (DEFAULT_TABLE table=test-table [id=76dc0574ce00478f934beed0dc7f62aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:12.594563 31357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d8c86be6fa46618a6c1d22da5dcef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:12.600293 31479 tablet_bootstrap.cc:492] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e: Bootstrap starting.
I20260810 07:24:12.602038 31481 tablet_bootstrap.cc:492] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c: Bootstrap starting.
I20260810 07:24:12.602038 31483 tablet_bootstrap.cc:492] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f: Bootstrap starting.
I20260810 07:24:12.603036 31479 tablet_bootstrap.cc:654] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:12.603847 31479 log.cc:824] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:12.604235 31481 tablet_bootstrap.cc:654] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:12.604238 31483 tablet_bootstrap.cc:654] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:12.605175 31481 log.cc:824] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:12.605473 31483 log.cc:824] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:12.606343 31479 tablet_bootstrap.cc:492] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e: No bootstrap required, opened a new log
I20260810 07:24:12.606550 31479 ts_tablet_manager.cc:1397] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:24:12.607121 31481 tablet_bootstrap.cc:492] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c: No bootstrap required, opened a new log
I20260810 07:24:12.607326 31481 ts_tablet_manager.cc:1397] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:24:12.607486 31483 tablet_bootstrap.cc:492] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f: No bootstrap required, opened a new log
I20260810 07:24:12.607679 31483 ts_tablet_manager.cc:1397] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:24:12.610675 31479 raft_consensus.cc:348] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36005 } } peers { permanent_uuid: "cf83e53225d447b098c47a9cc4b84a2e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46189 } } peers { permanent_uuid: "45c403e832f44b5d9838eea2204e487c" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33381 } }
I20260810 07:24:12.610983 31479 raft_consensus.cc:374] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:12.610816 31481 raft_consensus.cc:348] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36005 } } peers { permanent_uuid: "cf83e53225d447b098c47a9cc4b84a2e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46189 } } peers { permanent_uuid: "45c403e832f44b5d9838eea2204e487c" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33381 } }
I20260810 07:24:12.611045 31479 raft_consensus.cc:729] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf83e53225d447b098c47a9cc4b84a2e, State: Initialized, Role: FOLLOWER
I20260810 07:24:12.611056 31481 raft_consensus.cc:374] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:12.611112 31481 raft_consensus.cc:729] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c403e832f44b5d9838eea2204e487c, State: Initialized, Role: FOLLOWER
I20260810 07:24:12.611581 31481 consensus_queue.cc:260] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36005 } } peers { permanent_uuid: "cf83e53225d447b098c47a9cc4b84a2e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46189 } } peers { permanent_uuid: "45c403e832f44b5d9838eea2204e487c" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33381 } }
I20260810 07:24:12.611577 31479 consensus_queue.cc:260] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36005 } } peers { permanent_uuid: "cf83e53225d447b098c47a9cc4b84a2e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46189 } } peers { permanent_uuid: "45c403e832f44b5d9838eea2204e487c" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33381 } }
I20260810 07:24:12.611795 31483 raft_consensus.cc:348] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36005 } } peers { permanent_uuid: "cf83e53225d447b098c47a9cc4b84a2e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46189 } } peers { permanent_uuid: "45c403e832f44b5d9838eea2204e487c" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33381 } }
I20260810 07:24:12.612056 31483 raft_consensus.cc:374] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:12.612133 31483 raft_consensus.cc:729] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca8b4a56225744e4bbb66fc51b95e74f, State: Initialized, Role: FOLLOWER
I20260810 07:24:12.612532 31483 consensus_queue.cc:260] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36005 } } peers { permanent_uuid: "cf83e53225d447b098c47a9cc4b84a2e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46189 } } peers { permanent_uuid: "45c403e832f44b5d9838eea2204e487c" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33381 } }
I20260810 07:24:12.612679 31479 ts_tablet_manager.cc:1428] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 07:24:12.612720 31481 ts_tablet_manager.cc:1428] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:24:12.612820 31045 heartbeater.cc:503] Master 127.1.125.254:41323 was elected leader, sending a full tablet report...
I20260810 07:24:12.613016 31257 heartbeater.cc:503] Master 127.1.125.254:41323 was elected leader, sending a full tablet report...
I20260810 07:24:12.613510 31483 ts_tablet_manager.cc:1428] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:24:12.613811 31451 heartbeater.cc:503] Master 127.1.125.254:41323 was elected leader, sending a full tablet report...
I20260810 07:24:12.654816 31492 raft_consensus.cc:482] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:24:12.655016 31492 raft_consensus.cc:504] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36005 } } peers { permanent_uuid: "cf83e53225d447b098c47a9cc4b84a2e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46189 } } peers { permanent_uuid: "45c403e832f44b5d9838eea2204e487c" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33381 } }
I20260810 07:24:12.656253 31492 leader_election.cc:288] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf83e53225d447b098c47a9cc4b84a2e (127.1.125.194:46189), 45c403e832f44b5d9838eea2204e487c (127.1.125.193:33381)
I20260810 07:24:12.660694 31478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.656683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:54797 (local address 127.1.125.194:46189)
0810 07:24:12.656807 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:12.656813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:12.656828 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:24:12.656902 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:12.656906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:12.656960 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:12.657047 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:12.657351 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.657927 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.659470 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.660079 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.660237 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.660300 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:12.660449 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 07:24:12.660549 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:24:12.660699 31453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.656424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:46189 (local address 127.1.125.195:54797)
0810 07:24:12.656691 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:12.656709 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:12.656808 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:12.657060 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:12.657063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:12.657082 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:12.657229 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.657235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.658031 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.658036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.659342 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.659354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.660221 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.660224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.660308 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.660329 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:24:12.660432 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 07:24:12.660534 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":1}
I20260810 07:24:12.660923 31476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.656660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:47281 (local address 127.1.125.193:33381)
0810 07:24:12.656794 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:12.656799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:12.657068 (+ 269us) server_negotiation.cc:407] Connection header received
0810 07:24:12.657250 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:12.657253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:12.657309 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:12.657394 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:12.657667 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.658327 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.659698 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.660335 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.660450 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.660522 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:12.660696 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 07:24:12.660781 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:24:12.660926 31493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.656540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:33381 (local address 127.1.125.195:47281)
0810 07:24:12.656970 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:12.656982 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:12.657139 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:12.657404 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:12.657407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:12.657424 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:12.657556 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.657562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.658447 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.658451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.659584 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:12.659593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.660452 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.660455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:12.660570 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.660592 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:24:12.660666 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:24:12.660731 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":26,"spinlock_wait_cycles":38016,"thread_start_us":151,"threads_started":1}
I20260810 07:24:12.661432 31186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d8c86be6fa46618a6c1d22da5dcef7" candidate_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf83e53225d447b098c47a9cc4b84a2e" is_pre_election: true
I20260810 07:24:12.661749 31186 raft_consensus.cc:2454] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca8b4a56225744e4bbb66fc51b95e74f in term 0.
I20260810 07:24:12.661928 30973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d8c86be6fa46618a6c1d22da5dcef7" candidate_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c403e832f44b5d9838eea2204e487c" is_pre_election: true
I20260810 07:24:12.662243 30973 raft_consensus.cc:2454] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca8b4a56225744e4bbb66fc51b95e74f in term 0.
I20260810 07:24:12.662842 31292 leader_election.cc:302] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 45c403e832f44b5d9838eea2204e487c, ca8b4a56225744e4bbb66fc51b95e74f, cf83e53225d447b098c47a9cc4b84a2e; no voters:
I20260810 07:24:12.663174 31492 raft_consensus.cc:2781] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:24:12.663267 31492 raft_consensus.cc:482] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:24:12.663331 31492 raft_consensus.cc:3037] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:12.664371 31492 raft_consensus.cc:504] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36005 } } peers { permanent_uuid: "cf83e53225d447b098c47a9cc4b84a2e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46189 } } peers { permanent_uuid: "45c403e832f44b5d9838eea2204e487c" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33381 } }
I20260810 07:24:12.664808 31492 leader_election.cc:288] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [CANDIDATE]: Term 1 election: Requested vote from peers cf83e53225d447b098c47a9cc4b84a2e (127.1.125.194:46189), 45c403e832f44b5d9838eea2204e487c (127.1.125.193:33381)
I20260810 07:24:12.665135 31186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d8c86be6fa46618a6c1d22da5dcef7" candidate_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf83e53225d447b098c47a9cc4b84a2e"
I20260810 07:24:12.665237 30973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d8c86be6fa46618a6c1d22da5dcef7" candidate_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c403e832f44b5d9838eea2204e487c"
I20260810 07:24:12.665256 31186 raft_consensus.cc:3037] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:12.665380 30973 raft_consensus.cc:3037] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:12.666368 31186 raft_consensus.cc:2454] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca8b4a56225744e4bbb66fc51b95e74f in term 1.
I20260810 07:24:12.666507 30973 raft_consensus.cc:2454] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca8b4a56225744e4bbb66fc51b95e74f in term 1.
I20260810 07:24:12.666694 31292 leader_election.cc:302] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca8b4a56225744e4bbb66fc51b95e74f, cf83e53225d447b098c47a9cc4b84a2e; no voters:
I20260810 07:24:12.666858 31492 raft_consensus.cc:2781] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:12.667101 31492 raft_consensus.cc:686] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [term 1 LEADER]: Becoming Leader. State: Replica: ca8b4a56225744e4bbb66fc51b95e74f, State: Running, Role: LEADER
I20260810 07:24:12.667424 31492 consensus_queue.cc:237] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36005 } } peers { permanent_uuid: "cf83e53225d447b098c47a9cc4b84a2e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46189 } } peers { permanent_uuid: "45c403e832f44b5d9838eea2204e487c" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33381 } }
I20260810 07:24:12.670599 30751 catalog_manager.cc:5495] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f reported cstate change: term changed from 0 to 1, leader changed from <none> to ca8b4a56225744e4bbb66fc51b95e74f (127.1.125.195). New cstate: current_term: 1 leader_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b4a56225744e4bbb66fc51b95e74f" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 36005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf83e53225d447b098c47a9cc4b84a2e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45c403e832f44b5d9838eea2204e487c" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33381 } health_report { overall_health: UNKNOWN } } }
I20260810 07:24:12.686134 31473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.682312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52332 (local address 127.1.125.195:36005)
0810 07:24:12.682506 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:12.682510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:12.682526 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:24:12.682678 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:12.682681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:12.682737 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:12.682856 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:12.683141 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.683656 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.684582 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:12.684799 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:12.684914 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:12.685256 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:12.685698 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:12.685846 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:12.685889 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 07:24:12.685952 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 07:24:12.692101 30973 raft_consensus.cc:1261] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c [term 1 FOLLOWER]: Refusing update from remote peer ca8b4a56225744e4bbb66fc51b95e74f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:12.692116 31186 raft_consensus.cc:1261] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e [term 1 FOLLOWER]: Refusing update from remote peer ca8b4a56225744e4bbb66fc51b95e74f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:12.692857 31494 consensus_queue.cc:1035] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf83e53225d447b098c47a9cc4b84a2e" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 46189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:12.693048 31492 consensus_queue.cc:1035] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [LEADER]: Connected to new peer: Peer: permanent_uuid: "45c403e832f44b5d9838eea2204e487c" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 33381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:12.700727 31497 mvcc.cc:205] Tried to move back new op lower bound from 7316875889419567104 to 7316875889328615424. Current Snapshot: MvccSnapshot[applied={T|T < 7316875889419567104}]
I20260810 07:24:12.702471 31498 mvcc.cc:205] Tried to move back new op lower bound from 7316875889419567104 to 7316875889328615424. Current Snapshot: MvccSnapshot[applied={T|T < 7316875889419567104}]
I20260810 07:24:12.703644 31501 mvcc.cc:205] Tried to move back new op lower bound from 7316875889419567104 to 7316875889328615424. Current Snapshot: MvccSnapshot[applied={T|T < 7316875889419567104}]
I20260810 07:24:12.709928 30749 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:24:12.710119 30749 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:24:12.712610 30749 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6e4e25502884dda816a070945d0557b: Sending DeleteTablet for 3 replicas of tablet 72d8c86be6fa46618a6c1d22da5dcef7
I20260810 07:24:12.713382 31357 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d8c86be6fa46618a6c1d22da5dcef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:12 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:52330
I20260810 07:24:12.713439 31156 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d8c86be6fa46618a6c1d22da5dcef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:12 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:46594
I20260810 07:24:12.713580 30946 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d8c86be6fa46618a6c1d22da5dcef7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:12 UTC) from {username='kudu', principal='kudu/127.1.125.254@KRBTEST.COM'} at 127.0.0.1:41340
I20260810 07:24:12.713930 31517 tablet_replica.cc:330] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e: stopping tablet replica
I20260810 07:24:12.714102 31518 tablet_replica.cc:330] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c: stopping tablet replica
I20260810 07:24:12.714250 31517 raft_consensus.cc:2229] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:24:12.714512 31518 raft_consensus.cc:2229] T 72d8c86be6fa46618a6c1d22da5dcef7 P 45c403e832f44b5d9838eea2204e487c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:24:12.714582 31517 raft_consensus.cc:2258] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:24:12.714969 31516 tablet_replica.cc:330] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f: stopping tablet replica
I20260810 07:24:12.715075 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 30848
I20260810 07:24:12.715274 31516 raft_consensus.cc:2229] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:24:12.715667 31516 raft_consensus.cc:2258] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:24:12.716454 31517 ts_tablet_manager.cc:1905] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:24:12.717382 31516 ts_tablet_manager.cc:1905] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:24:12.719354 31517 ts_tablet_manager.cc:1918] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:24:12.719441 31517 log.cc:1196] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/72d8c86be6fa46618a6c1d22da5dcef7
I20260810 07:24:12.719797 31517 ts_tablet_manager.cc:1939] T 72d8c86be6fa46618a6c1d22da5dcef7 P cf83e53225d447b098c47a9cc4b84a2e: Deleting consensus metadata
I20260810 07:24:12.721065 30735 catalog_manager.cc:4841] TS cf83e53225d447b098c47a9cc4b84a2e (127.1.125.194:46189): tablet 72d8c86be6fa46618a6c1d22da5dcef7 (table test-table [id=76dc0574ce00478f934beed0dc7f62aa]) successfully deleted
I20260810 07:24:12.721390 31516 ts_tablet_manager.cc:1918] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:24:12.721508 31516 log.cc:1196] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/72d8c86be6fa46618a6c1d22da5dcef7
I20260810 07:24:12.721854 31516 ts_tablet_manager.cc:1939] T 72d8c86be6fa46618a6c1d22da5dcef7 P ca8b4a56225744e4bbb66fc51b95e74f: Deleting consensus metadata
I20260810 07:24:12.725065 30736 catalog_manager.cc:4841] TS ca8b4a56225744e4bbb66fc51b95e74f (127.1.125.195:36005): tablet 72d8c86be6fa46618a6c1d22da5dcef7 (table test-table [id=76dc0574ce00478f934beed0dc7f62aa]) successfully deleted
W20260810 07:24:12.733067 30730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:33381 (error 108)
I20260810 07:24:12.734107 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 31051
I20260810 07:24:12.734369 31469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:12.733981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:33381 (local address 127.0.0.1:41350)
0810 07:24:12.734166 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:12.734254 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260810 07:24:12.736583 30730 catalog_manager.cc:4584] TS 45c403e832f44b5d9838eea2204e487c (127.1.125.193:33381): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72d8c86be6fa46618a6c1d22da5dcef7: Network error: Client connection negotiation failed: client connection to 127.1.125.193:33381: connect: Connection refused (error 111)
I20260810 07:24:12.745939 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 31262
I20260810 07:24:12.759184 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 30708
2026-08-10T07:24:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2675 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T07:24:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:12Z Disabled control of system clock
I20260810 07:24:12.798533 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38629
--webserver_interface=127.1.125.254
--webserver_port=0
--builtin_ntp_servers=127.1.125.212:43253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:38629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:24:12.911990 31524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:12.912262 31524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:12.912297 31524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:12.916288 31524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:12.916347 31524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:12.916361 31524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:12.916378 31524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:12.916392 31524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:12.921553 31524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:38629
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:12.922617 31524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:12.924048 31524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:12.932428 31530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:12.932612 31531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:12.932612 31534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:12.932890 31524 server_base.cc:1034] running on GCE node
I20260810 07:24:12.933440 31524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:12.934631 31524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:12.935832 31524 hybrid_clock.cc:648] HybridClock initialized: now 1786346652935811 us; error 30 us; skew 500 ppm
I20260810 07:24:12.942976 31524 webserver.cc:460] Webserver started at http://127.1.125.254:39679/ using document root <none> and password file <none>
I20260810 07:24:12.943629 31524 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:12.943729 31524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:12.943976 31524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:12.946187 31524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "abae00fe00084b7ea848063341269ab8"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-44d2"
server_key: "120d96d4b03d8b65505cbc60b8bd0d5b"
server_key_iv: "cc1f59d63b538364c720fa5bec883ee7"
server_key_version: "encryptionkey@0"
I20260810 07:24:12.946767 31524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "abae00fe00084b7ea848063341269ab8"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-44d2"
server_key: "120d96d4b03d8b65505cbc60b8bd0d5b"
server_key_iv: "cc1f59d63b538364c720fa5bec883ee7"
server_key_version: "encryptionkey@0"
I20260810 07:24:12.951728 31524 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 07:24:12.954381 31542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:12.955674 31524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:24:12.955931 31524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "abae00fe00084b7ea848063341269ab8"
format_stamp: "Formatted at 2026-08-10 07:24:12 on dist-test-slave-44d2"
server_key: "120d96d4b03d8b65505cbc60b8bd0d5b"
server_key_iv: "cc1f59d63b538364c720fa5bec883ee7"
server_key_version: "encryptionkey@0"
I20260810 07:24:12.956099 31524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:13.007054 31524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:13.007915 31524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:13.008152 31524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:13.016179 31620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:38629 every 8 connection(s)
I20260810 07:24:13.016184 31524 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:38629
I20260810 07:24:13.017338 31524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:13.020612 31621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:13.025584 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 31524
I20260810 07:24:13.025759 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:13.026086 1527 external_mini_cluster.cc:1428] Setting key 3827bcfe9a17a14f7a76964a92972771
I20260810 07:24:13.026117 31621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8: Bootstrap starting.
I20260810 07:24:13.028800 31621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:13.029562 31621 log.cc:824] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:13.031430 31621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8: No bootstrap required, opened a new log
I20260810 07:24:13.035025 31621 raft_consensus.cc:348] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abae00fe00084b7ea848063341269ab8" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38629 } }
I20260810 07:24:13.035269 31621 raft_consensus.cc:374] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:13.035326 31621 raft_consensus.cc:729] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abae00fe00084b7ea848063341269ab8, State: Initialized, Role: FOLLOWER
I20260810 07:24:13.035843 31621 consensus_queue.cc:260] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abae00fe00084b7ea848063341269ab8" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38629 } }
I20260810 07:24:13.036005 31621 raft_consensus.cc:388] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:13.036077 31621 raft_consensus.cc:482] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:13.036177 31621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:13.036909 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.027612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.1.125.254:38629)
0810 07:24:13.028414 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:13.028429 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:13.028468 (+ 39us) server_negotiation.cc:407] Connection header received
0810 07:24:13.029457 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:13.029477 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:13.029942 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:13.030299 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:13.030765 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.031811 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.032924 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.033432 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.033622 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.034655 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:13.034660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:13.034673 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:13.035095 (+ 422us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:13.035158 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:13.035181 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:13.035307 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:13.035508 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 07:24:13.035815 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":123,"threads_started":1}
I20260810 07:24:13.037315 31621 raft_consensus.cc:504] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abae00fe00084b7ea848063341269ab8" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38629 } }
I20260810 07:24:13.037674 31621 leader_election.cc:302] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abae00fe00084b7ea848063341269ab8; no voters:
I20260810 07:24:13.038019 31621 leader_election.cc:288] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:13.038388 31631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:13.038682 31631 raft_consensus.cc:686] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [term 1 LEADER]: Becoming Leader. State: Replica: abae00fe00084b7ea848063341269ab8, State: Running, Role: LEADER
I20260810 07:24:13.039113 31631 consensus_queue.cc:237] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abae00fe00084b7ea848063341269ab8" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38629 } }
I20260810 07:24:13.039283 31621 sys_catalog.cc:564] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:13.040737 31633 sys_catalog.cc:455] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abae00fe00084b7ea848063341269ab8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abae00fe00084b7ea848063341269ab8" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38629 } } }
I20260810 07:24:13.040768 31632 sys_catalog.cc:455] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abae00fe00084b7ea848063341269ab8. Latest consensus state: current_term: 1 leader_uuid: "abae00fe00084b7ea848063341269ab8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abae00fe00084b7ea848063341269ab8" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38629 } } }
I20260810 07:24:13.040900 31633 sys_catalog.cc:458] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:13.040920 31632 sys_catalog.cc:458] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:13.041386 31645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:13.044965 31645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:13.051404 31645 catalog_manager.cc:1347] Generated new cluster ID: 139b7305cd51450b99e40f6f3dc8c515
I20260810 07:24:13.051488 31645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:13.072556 31645 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:13.073550 31645 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:13.083240 31645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8: Generated new TSK 0
I20260810 07:24:13.084100 31645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:13.090322 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.193:0
--local_ip_for_outbound_sockets=127.1.125.193
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38629
--builtin_ntp_servers=127.1.125.212:43253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:24:13.209158 31656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:13.209436 31656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:13.209503 31656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:13.214214 31656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:13.214269 31656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:13.214325 31656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.193
I20260810 07:24:13.219309 31656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.125.193
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.193
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:13.220396 31656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:13.222160 31656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:13.232213 31661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:13.232213 31667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:13.232319 31663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:13.232372 31656 server_base.cc:1034] running on GCE node
I20260810 07:24:13.232908 31656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:13.233608 31656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:13.234808 31656 hybrid_clock.cc:648] HybridClock initialized: now 1786346653234795 us; error 31 us; skew 500 ppm
I20260810 07:24:13.236738 31656 webserver.cc:460] Webserver started at http://127.1.125.193:33951/ using document root <none> and password file <none>
I20260810 07:24:13.237339 31656 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:13.237437 31656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:13.237671 31656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:13.239867 31656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/data/instance:
uuid: "cbf4b88ebffa4d869e39e4d1f2d34377"
format_stamp: "Formatted at 2026-08-10 07:24:13 on dist-test-slave-44d2"
server_key: "964f863216ed3ef2b61a05b451064747"
server_key_iv: "0aca006e10e4d6555b24ed29b2263273"
server_key_version: "encryptionkey@0"
I20260810 07:24:13.240453 31656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance:
uuid: "cbf4b88ebffa4d869e39e4d1f2d34377"
format_stamp: "Formatted at 2026-08-10 07:24:13 on dist-test-slave-44d2"
server_key: "964f863216ed3ef2b61a05b451064747"
server_key_iv: "0aca006e10e4d6555b24ed29b2263273"
server_key_version: "encryptionkey@0"
I20260810 07:24:13.244340 31656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260810 07:24:13.246811 31676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:13.248006 31656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:24:13.248169 31656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/wal
uuid: "cbf4b88ebffa4d869e39e4d1f2d34377"
format_stamp: "Formatted at 2026-08-10 07:24:13 on dist-test-slave-44d2"
server_key: "964f863216ed3ef2b61a05b451064747"
server_key_iv: "0aca006e10e4d6555b24ed29b2263273"
server_key_version: "encryptionkey@0"
I20260810 07:24:13.248286 31656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:13.260702 31656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:13.261514 31656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:13.261724 31656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:13.262342 31656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:13.263335 31656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:13.263413 31656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:13.263482 31656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:13.263535 31656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:13.273017 31656 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.193:39517
I20260810 07:24:13.273059 31846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.193:39517 every 8 connection(s)
I20260810 07:24:13.273962 31656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/data/info.pb
I20260810 07:24:13.278504 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 31656
I20260810 07:24:13.278640 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-0/wal/instance
I20260810 07:24:13.279031 1527 external_mini_cluster.cc:1428] Setting key bc65ac183cc714d89c302f9e7b2c6d6d
I20260810 07:24:13.281064 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.194:0
--local_ip_for_outbound_sockets=127.1.125.194
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38629
--builtin_ntp_servers=127.1.125.212:43253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:24:13.283946 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.275920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.193:42315 (local address 127.1.125.254:38629)
0810 07:24:13.276068 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:13.276072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:13.277190 (+ 1118us) server_negotiation.cc:407] Connection header received
0810 07:24:13.278772 (+ 1582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:13.278777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:13.278834 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:13.278910 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:13.280066 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.280686 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.281952 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.282187 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.282304 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.283440 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:13.283443 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:13.283446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:13.283467 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:13.283486 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:13.283496 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:13.283578 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:13.283739 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 07:24:13.283804 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 07:24:13.284865 31850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.276334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38629 (local address 127.1.125.193:42315)
0810 07:24:13.277026 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:13.277066 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:13.278499 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:13.279130 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:13.279139 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:13.279248 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:13.279921 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.279938 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.280828 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.280831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.281821 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.281831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.282348 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.282352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.282522 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.283176 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:24:13.283177 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:13.283218 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:24:13.283283 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:13.283581 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:13.283594 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:13.283600 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:24:13.283739 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 07:24:13.284026 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":149,"threads_started":1}
I20260810 07:24:13.286046 31847 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38629
I20260810 07:24:13.286280 31847 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:13.286803 31847 heartbeater.cc:511] Master 127.1.125.254:38629 requested a full tablet report, sending...
I20260810 07:24:13.288354 31568 ts_manager.cc:194] Registered new tserver with Master: cbf4b88ebffa4d869e39e4d1f2d34377 (127.1.125.193:39517)
I20260810 07:24:13.289680 31568 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.125.193:42315
W20260810 07:24:13.400274 31851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:13.400620 31851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:13.400720 31851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:13.404830 31851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:13.404911 31851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:13.405001 31851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.194
I20260810 07:24:13.409754 31851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.125.194
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.194
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:13.410833 31851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:13.412261 31851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:13.422433 31861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:13.422541 31851 server_base.cc:1034] running on GCE node
W20260810 07:24:13.422470 31859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:13.422487 31858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:13.423208 31851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:13.423868 31851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:13.425065 31851 hybrid_clock.cc:648] HybridClock initialized: now 1786346653425029 us; error 47 us; skew 500 ppm
I20260810 07:24:13.426941 31851 webserver.cc:460] Webserver started at http://127.1.125.194:42889/ using document root <none> and password file <none>
I20260810 07:24:13.427561 31851 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:13.427628 31851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:13.427894 31851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:13.430158 31851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/data/instance:
uuid: "ffd56028a9194e7591d28826190ab3c5"
format_stamp: "Formatted at 2026-08-10 07:24:13 on dist-test-slave-44d2"
server_key: "53ba2775959a778b5173a39f3a5f7b91"
server_key_iv: "3bc867d100ab14b6b10434d78fe3108d"
server_key_version: "encryptionkey@0"
I20260810 07:24:13.430766 31851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance:
uuid: "ffd56028a9194e7591d28826190ab3c5"
format_stamp: "Formatted at 2026-08-10 07:24:13 on dist-test-slave-44d2"
server_key: "53ba2775959a778b5173a39f3a5f7b91"
server_key_iv: "3bc867d100ab14b6b10434d78fe3108d"
server_key_version: "encryptionkey@0"
I20260810 07:24:13.434633 31851 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:24:13.437119 31871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:13.438354 31851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:24:13.438493 31851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/wal
uuid: "ffd56028a9194e7591d28826190ab3c5"
format_stamp: "Formatted at 2026-08-10 07:24:13 on dist-test-slave-44d2"
server_key: "53ba2775959a778b5173a39f3a5f7b91"
server_key_iv: "3bc867d100ab14b6b10434d78fe3108d"
server_key_version: "encryptionkey@0"
I20260810 07:24:13.438652 31851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:13.479909 31851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:13.480762 31851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:13.480976 31851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:13.481631 31851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:13.482637 31851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:13.482713 31851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:13.482781 31851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:13.482831 31851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:13.491940 31851 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.194:44369
I20260810 07:24:13.491966 32028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.194:44369 every 8 connection(s)
I20260810 07:24:13.492925 31851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/data/info.pb
I20260810 07:24:13.499552 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 31851
I20260810 07:24:13.499688 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/wal/instance
I20260810 07:24:13.499989 1527 external_mini_cluster.cc:1428] Setting key 79900d5fbfb05da17b5989b5107551bb
I20260810 07:24:13.502206 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.125.195:0
--local_ip_for_outbound_sockets=127.1.125.195
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38629
--builtin_ntp_servers=127.1.125.212:43253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:24:13.502832 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.494792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.194:39113 (local address 127.1.125.254:38629)
0810 07:24:13.494943 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:13.494948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:13.496053 (+ 1105us) server_negotiation.cc:407] Connection header received
0810 07:24:13.497502 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:13.497505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:13.497554 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:13.497630 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:13.498667 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.499195 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.500562 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.500799 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.500881 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.502188 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:13.502192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:13.502195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:13.502209 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:13.502233 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:13.502245 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:13.502336 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:13.502542 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 07:24:13.502624 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:24:13.503603 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.495248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38629 (local address 127.1.125.194:39113)
0810 07:24:13.495898 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:13.495940 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:13.497308 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:13.497873 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:13.497890 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:13.498011 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:13.498524 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.498538 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.499304 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.499306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.500358 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.500375 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.500952 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.500957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.501172 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.501905 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:24:13.501907 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:13.501948 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:24:13.502019 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:13.502356 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:13.502369 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:13.502377 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:24:13.502526 (+ 149us) client_negotiation.cc:240] Negotiation successful
0810 07:24:13.502754 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":113,"threads_started":1}
I20260810 07:24:13.504928 32029 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38629
I20260810 07:24:13.505187 32029 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:13.505710 32029 heartbeater.cc:511] Master 127.1.125.254:38629 requested a full tablet report, sending...
I20260810 07:24:13.506814 31568 ts_manager.cc:194] Registered new tserver with Master: ffd56028a9194e7591d28826190ab3c5 (127.1.125.194:44369)
I20260810 07:24:13.507694 31568 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.125.194:39113
W20260810 07:24:13.618736 32033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:13.618971 32033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:13.619009 32033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:13.622942 32033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:13.622995 32033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:13.623049 32033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.125.195
I20260810 07:24:13.627650 32033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.125.195
--webserver_port=0
--tserver_master_addrs=127.1.125.254:38629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.125.195
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:13.628693 32033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:13.629997 32033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:13.639576 32044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:13.639566 32042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:13.639546 32046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:13.640076 32033 server_base.cc:1034] running on GCE node
I20260810 07:24:13.640576 32033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:13.641192 32033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:13.642393 32033 hybrid_clock.cc:648] HybridClock initialized: now 1786346653642375 us; error 30 us; skew 500 ppm
I20260810 07:24:13.644232 32033 webserver.cc:460] Webserver started at http://127.1.125.195:42527/ using document root <none> and password file <none>
I20260810 07:24:13.644883 32033 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:13.644948 32033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:13.645187 32033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:13.647281 32033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/data/instance:
uuid: "d4b13d8f9e2f4824937a761d438a2a7c"
format_stamp: "Formatted at 2026-08-10 07:24:13 on dist-test-slave-44d2"
server_key: "d122e99da4f9f79b61d8d71469fde660"
server_key_iv: "79abccfc9cd67354c73e54542c1682e5"
server_key_version: "encryptionkey@0"
I20260810 07:24:13.647850 32033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance:
uuid: "d4b13d8f9e2f4824937a761d438a2a7c"
format_stamp: "Formatted at 2026-08-10 07:24:13 on dist-test-slave-44d2"
server_key: "d122e99da4f9f79b61d8d71469fde660"
server_key_iv: "79abccfc9cd67354c73e54542c1682e5"
server_key_version: "encryptionkey@0"
I20260810 07:24:13.651582 32033 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:24:13.654102 32058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:13.655263 32033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:24:13.655401 32033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/wal
uuid: "d4b13d8f9e2f4824937a761d438a2a7c"
format_stamp: "Formatted at 2026-08-10 07:24:13 on dist-test-slave-44d2"
server_key: "d122e99da4f9f79b61d8d71469fde660"
server_key_iv: "79abccfc9cd67354c73e54542c1682e5"
server_key_version: "encryptionkey@0"
I20260810 07:24:13.655524 32033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:13.671166 32033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:13.671804 32033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:13.672009 32033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:13.672644 32033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:24:13.673607 32033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:24:13.673682 32033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:13.673755 32033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:24:13.673807 32033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:13.682968 32033 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.195:40341
I20260810 07:24:13.683019 32216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.195:40341 every 8 connection(s)
I20260810 07:24:13.683897 32033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/data/info.pb
I20260810 07:24:13.690578 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 32033
I20260810 07:24:13.690713 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/wal/instance
I20260810 07:24:13.691007 1527 external_mini_cluster.cc:1428] Setting key fb08c3b78ed3ddb14bf2fd3e43d7cc4a
I20260810 07:24:13.693267 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.685753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:40487 (local address 127.1.125.254:38629)
0810 07:24:13.685891 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:13.685894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:13.686955 (+ 1061us) server_negotiation.cc:407] Connection header received
0810 07:24:13.688363 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:13.688366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:13.688413 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:13.688513 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:13.689370 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.689891 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.691155 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.691391 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.691509 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.692748 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:13.692752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:13.692755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:13.692774 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:13.692792 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:13.692800 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:13.692883 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:13.693049 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 07:24:13.693118 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:24:13.694149 32223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.686161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:38629 (local address 127.1.125.195:40487)
0810 07:24:13.686806 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:13.686847 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:13.688164 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:13.688661 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:13.688667 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:13.688751 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:13.689248 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.689261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.690004 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.690006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.691011 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.691022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.691556 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.691560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.691742 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.692408 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:24:13.692409 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:13.692451 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:24:13.692573 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:13.692885 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:13.692898 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:13.692907 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 07:24:13.693046 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 07:24:13.693334 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":116,"threads_started":1}
I20260810 07:24:13.695214 32217 heartbeater.cc:347] Connected to a master server at 127.1.125.254:38629
I20260810 07:24:13.695453 32217 heartbeater.cc:464] Registering TS with master...
I20260810 07:24:13.695971 32217 heartbeater.cc:511] Master 127.1.125.254:38629 requested a full tablet report, sending...
I20260810 07:24:13.697005 31568 ts_manager.cc:194] Registered new tserver with Master: d4b13d8f9e2f4824937a761d438a2a7c (127.1.125.195:40341)
I20260810 07:24:13.697554 31568 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.125.195:40487
I20260810 07:24:13.704715 1527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:24:13.710505 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.706671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.1.125.254:38629)
0810 07:24:13.706823 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:13.706828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:13.706975 (+ 147us) server_negotiation.cc:407] Connection header received
0810 07:24:13.707129 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:13.707132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:13.707174 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:13.707263 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:13.707552 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.708051 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.708936 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.709168 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.709273 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.710048 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:13.710052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:13.710055 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:13.710067 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:13.710085 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:13.710101 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:13.710179 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:13.710303 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 07:24:13.710355 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 07:24:13.713126 31568 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:24:13.716114 31568 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:24:13.731027 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.725541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.1.125.194:44369)
0810 07:24:13.726232 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:13.726245 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:13.726269 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:24:13.726335 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:13.726340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:13.726482 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:13.726628 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:13.727137 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.727876 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.729558 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.730299 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.730436 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.730648 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:13.730749 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 07:24:13.730849 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":165,"threads_started":1}
I20260810 07:24:13.731021 32238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.725133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:44369 (local address 127.0.0.1:34762)
0810 07:24:13.726106 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:13.726119 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:13.726194 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:13.726784 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:13.726787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:13.726816 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:13.726984 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.726989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.727993 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.727997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.729416 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.729426 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.730437 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.730441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.730556 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.730596 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 07:24:13.730720 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 07:24:13.730847 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"mutex_wait_us":230,"spinlock_wait_cycles":6400,"thread_start_us":88,"threads_started":1}
I20260810 07:24:13.731529 32235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.724679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.195:40341 (local address 127.0.0.1:35964)
0810 07:24:13.725464 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:13.726030 (+ 566us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:13.726157 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:13.726784 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:13.726787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:13.726806 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:13.726980 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.726994 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.728332 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.728335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.729843 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.729851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.731118 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.731123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.731242 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.731264 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:24:13.731330 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 07:24:13.731392 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":267,"spinlock_wait_cycles":2816,"thread_start_us":83,"threads_started":1}
I20260810 07:24:13.731846 32240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.724949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:39517 (local address 127.0.0.1:35920)
0810 07:24:13.725778 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:13.725843 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:13.725970 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:13.726644 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:13.726650 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:13.726675 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:13.726820 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.726827 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.728455 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.728458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.729755 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.729767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.731454 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.731458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.731545 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.731578 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 07:24:13.731655 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 07:24:13.731728 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":92,"thread_start_us":119,"threads_started":1}
I20260810 07:24:13.732014 32237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.724811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35964 (local address 127.1.125.195:40341)
0810 07:24:13.725705 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:13.725717 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:13.726269 (+ 552us) server_negotiation.cc:407] Connection header received
0810 07:24:13.726355 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:13.726358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:13.726503 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:13.726670 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:13.727114 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.728193 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.729969 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.730753 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.730863 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.731044 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:13.731732 (+ 688us) server_negotiation.cc:299] Negotiation successful
0810 07:24:13.731830 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":114,"threads_started":1}
I20260810 07:24:13.732416 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.725341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35920 (local address 127.1.125.193:39517)
0810 07:24:13.725929 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:13.725941 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:13.725993 (+ 52us) server_negotiation.cc:407] Connection header received
0810 07:24:13.726235 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:13.726243 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:13.726451 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:13.726622 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:13.726952 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.728339 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.729910 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.730753 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.730863 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.731045 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:13.732107 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0810 07:24:13.732244 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":157,"threads_started":1}
I20260810 07:24:13.734228 31940 tablet_service.cc:1467] Processing CreateTablet for tablet 300bc36a416f4c81aeb8e29a7498ea04 (DEFAULT_TABLE table=test-table [id=3372fca7584d49f4bac477a2c1f32476]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:13.735461 31940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 300bc36a416f4c81aeb8e29a7498ea04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:13.735169 31753 tablet_service.cc:1467] Processing CreateTablet for tablet 300bc36a416f4c81aeb8e29a7498ea04 (DEFAULT_TABLE table=test-table [id=3372fca7584d49f4bac477a2c1f32476]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:13.736151 31753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 300bc36a416f4c81aeb8e29a7498ea04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:13.735903 32124 tablet_service.cc:1467] Processing CreateTablet for tablet 300bc36a416f4c81aeb8e29a7498ea04 (DEFAULT_TABLE table=test-table [id=3372fca7584d49f4bac477a2c1f32476]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:24:13.736918 32124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 300bc36a416f4c81aeb8e29a7498ea04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:13.743007 32244 tablet_bootstrap.cc:492] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377: Bootstrap starting.
I20260810 07:24:13.743968 32246 tablet_bootstrap.cc:492] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5: Bootstrap starting.
I20260810 07:24:13.745558 32244 tablet_bootstrap.cc:654] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:13.745851 32246 tablet_bootstrap.cc:654] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:13.746371 32244 log.cc:824] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:13.746677 32246 log.cc:824] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:13.747396 32248 tablet_bootstrap.cc:492] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c: Bootstrap starting.
I20260810 07:24:13.748718 32246 tablet_bootstrap.cc:492] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5: No bootstrap required, opened a new log
I20260810 07:24:13.748718 32244 tablet_bootstrap.cc:492] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377: No bootstrap required, opened a new log
I20260810 07:24:13.748922 32246 ts_tablet_manager.cc:1397] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:24:13.748922 32244 ts_tablet_manager.cc:1397] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:24:13.749221 32248 tablet_bootstrap.cc:654] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:13.749877 32248 log.cc:824] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:13.751631 32248 tablet_bootstrap.cc:492] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c: No bootstrap required, opened a new log
I20260810 07:24:13.751834 32248 ts_tablet_manager.cc:1397] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 07:24:13.753497 32246 raft_consensus.cc:348] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40341 } } peers { permanent_uuid: "cbf4b88ebffa4d869e39e4d1f2d34377" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 39517 } } peers { permanent_uuid: "ffd56028a9194e7591d28826190ab3c5" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44369 } }
I20260810 07:24:13.753808 32246 raft_consensus.cc:374] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:13.753875 32246 raft_consensus.cc:729] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd56028a9194e7591d28826190ab3c5, State: Initialized, Role: FOLLOWER
I20260810 07:24:13.754345 32246 consensus_queue.cc:260] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40341 } } peers { permanent_uuid: "cbf4b88ebffa4d869e39e4d1f2d34377" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 39517 } } peers { permanent_uuid: "ffd56028a9194e7591d28826190ab3c5" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44369 } }
I20260810 07:24:13.753497 32244 raft_consensus.cc:348] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40341 } } peers { permanent_uuid: "cbf4b88ebffa4d869e39e4d1f2d34377" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 39517 } } peers { permanent_uuid: "ffd56028a9194e7591d28826190ab3c5" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44369 } }
I20260810 07:24:13.754557 32244 raft_consensus.cc:374] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:13.754642 32244 raft_consensus.cc:729] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf4b88ebffa4d869e39e4d1f2d34377, State: Initialized, Role: FOLLOWER
I20260810 07:24:13.755131 32029 heartbeater.cc:503] Master 127.1.125.254:38629 was elected leader, sending a full tablet report...
I20260810 07:24:13.755158 32244 consensus_queue.cc:260] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40341 } } peers { permanent_uuid: "cbf4b88ebffa4d869e39e4d1f2d34377" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 39517 } } peers { permanent_uuid: "ffd56028a9194e7591d28826190ab3c5" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44369 } }
I20260810 07:24:13.755493 32246 ts_tablet_manager.cc:1428] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 07:24:13.756055 32244 ts_tablet_manager.cc:1428] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 07:24:13.755898 32248 raft_consensus.cc:348] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40341 } } peers { permanent_uuid: "cbf4b88ebffa4d869e39e4d1f2d34377" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 39517 } } peers { permanent_uuid: "ffd56028a9194e7591d28826190ab3c5" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44369 } }
I20260810 07:24:13.756129 32248 raft_consensus.cc:374] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:13.756177 32248 raft_consensus.cc:729] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4b13d8f9e2f4824937a761d438a2a7c, State: Initialized, Role: FOLLOWER
I20260810 07:24:13.756405 31847 heartbeater.cc:503] Master 127.1.125.254:38629 was elected leader, sending a full tablet report...
I20260810 07:24:13.756640 32248 consensus_queue.cc:260] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40341 } } peers { permanent_uuid: "cbf4b88ebffa4d869e39e4d1f2d34377" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 39517 } } peers { permanent_uuid: "ffd56028a9194e7591d28826190ab3c5" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44369 } }
I20260810 07:24:13.757295 32217 heartbeater.cc:503] Master 127.1.125.254:38629 was elected leader, sending a full tablet report...
I20260810 07:24:13.757583 32248 ts_tablet_manager.cc:1428] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260810 07:24:13.775274 31849 tablet.cc:2369] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:24:13.934994 32219 tablet.cc:2369] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:24:13.973690 32255 raft_consensus.cc:482] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:24:13.973925 32255 raft_consensus.cc:504] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40341 } } peers { permanent_uuid: "cbf4b88ebffa4d869e39e4d1f2d34377" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 39517 } } peers { permanent_uuid: "ffd56028a9194e7591d28826190ab3c5" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44369 } }
I20260810 07:24:13.975203 32255 leader_election.cc:288] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbf4b88ebffa4d869e39e4d1f2d34377 (127.1.125.193:39517), ffd56028a9194e7591d28826190ab3c5 (127.1.125.194:44369)
I20260810 07:24:13.980093 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.975716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:43509 (local address 127.1.125.194:44369)
0810 07:24:13.975863 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:13.975867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:13.975884 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:24:13.975945 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:13.975950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:13.976013 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:13.976084 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:13.976518 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.977275 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.978834 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.979428 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.979558 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.979609 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:13.979884 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 07:24:13.979956 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:24:13.980093 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.975609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.125.195:38805 (local address 127.1.125.193:39517)
0810 07:24:13.975753 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:13.975758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:13.976217 (+ 459us) server_negotiation.cc:407] Connection header received
0810 07:24:13.976275 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:13.976277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:13.976321 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:13.976381 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:24:13.976752 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.977461 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.978833 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.979428 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.979559 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.979609 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:13.979884 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 07:24:13.979956 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 07:24:13.980103 32223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.975500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.194:44369 (local address 127.1.125.195:43509)
0810 07:24:13.975666 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:13.975686 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:13.975794 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:13.976149 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:13.976154 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:13.976179 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:13.976326 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.976332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.977388 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.977390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.978690 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.978703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.979562 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.979566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.979699 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.979762 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 07:24:13.979847 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 07:24:13.979940 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":84}
I20260810 07:24:13.980123 32256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:13.975518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:39517 (local address 127.1.125.195:38805)
0810 07:24:13.976052 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:13.976100 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:13.976198 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:13.976423 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:13.976426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:13.976449 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:24:13.976618 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.976623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.977561 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.977564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.978696 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:13.978702 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:13.979562 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:13.979566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:13.979694 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:13.979757 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 07:24:13.979847 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 07:24:13.979940 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":172,"thread_start_us":148,"threads_started":1}
I20260810 07:24:13.980998 31781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300bc36a416f4c81aeb8e29a7498ea04" candidate_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf4b88ebffa4d869e39e4d1f2d34377" is_pre_election: true
I20260810 07:24:13.980981 31967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300bc36a416f4c81aeb8e29a7498ea04" candidate_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd56028a9194e7591d28826190ab3c5" is_pre_election: true
I20260810 07:24:13.981341 31967 raft_consensus.cc:2454] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4b13d8f9e2f4824937a761d438a2a7c in term 0.
I20260810 07:24:13.981341 31781 raft_consensus.cc:2454] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4b13d8f9e2f4824937a761d438a2a7c in term 0.
I20260810 07:24:13.981930 32060 leader_election.cc:302] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: cbf4b88ebffa4d869e39e4d1f2d34377, d4b13d8f9e2f4824937a761d438a2a7c, ffd56028a9194e7591d28826190ab3c5; no voters:
I20260810 07:24:13.982266 32255 raft_consensus.cc:2781] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:24:13.982374 32255 raft_consensus.cc:482] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:24:13.982457 32255 raft_consensus.cc:3037] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:13.983598 32255 raft_consensus.cc:504] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40341 } } peers { permanent_uuid: "cbf4b88ebffa4d869e39e4d1f2d34377" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 39517 } } peers { permanent_uuid: "ffd56028a9194e7591d28826190ab3c5" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44369 } }
I20260810 07:24:13.983960 32255 leader_election.cc:288] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [CANDIDATE]: Term 1 election: Requested vote from peers cbf4b88ebffa4d869e39e4d1f2d34377 (127.1.125.193:39517), ffd56028a9194e7591d28826190ab3c5 (127.1.125.194:44369)
I20260810 07:24:13.984362 31781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300bc36a416f4c81aeb8e29a7498ea04" candidate_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf4b88ebffa4d869e39e4d1f2d34377"
I20260810 07:24:13.984375 31967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300bc36a416f4c81aeb8e29a7498ea04" candidate_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd56028a9194e7591d28826190ab3c5"
I20260810 07:24:13.984534 31781 raft_consensus.cc:3037] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:13.984534 31967 raft_consensus.cc:3037] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:13.985630 31781 raft_consensus.cc:2454] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4b13d8f9e2f4824937a761d438a2a7c in term 1.
I20260810 07:24:13.985661 31967 raft_consensus.cc:2454] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4b13d8f9e2f4824937a761d438a2a7c in term 1.
I20260810 07:24:13.985994 32060 leader_election.cc:302] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbf4b88ebffa4d869e39e4d1f2d34377, d4b13d8f9e2f4824937a761d438a2a7c; no voters:
I20260810 07:24:13.986178 32255 raft_consensus.cc:2781] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:13.986389 32255 raft_consensus.cc:686] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [term 1 LEADER]: Becoming Leader. State: Replica: d4b13d8f9e2f4824937a761d438a2a7c, State: Running, Role: LEADER
I20260810 07:24:13.986727 32255 consensus_queue.cc:237] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40341 } } peers { permanent_uuid: "cbf4b88ebffa4d869e39e4d1f2d34377" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 39517 } } peers { permanent_uuid: "ffd56028a9194e7591d28826190ab3c5" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44369 } }
I20260810 07:24:13.990108 31568 catalog_manager.cc:5495] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c reported cstate change: term changed from 0 to 1, leader changed from <none> to d4b13d8f9e2f4824937a761d438a2a7c (127.1.125.195). New cstate: current_term: 1 leader_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b13d8f9e2f4824937a761d438a2a7c" member_type: VOTER last_known_addr { host: "127.1.125.195" port: 40341 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbf4b88ebffa4d869e39e4d1f2d34377" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 39517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffd56028a9194e7591d28826190ab3c5" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44369 } health_report { overall_health: UNKNOWN } } }
W20260810 07:24:13.994047 32030 tablet.cc:2369] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:24:14.023021 32237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.019292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35970 (local address 127.1.125.195:40341)
0810 07:24:14.019427 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:14.019431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:14.019444 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:24:14.019519 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:14.019521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:14.019574 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:14.019665 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:14.019940 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.020446 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.021649 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:14.021859 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:14.021932 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:14.022234 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:14.022609 (+ 375us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:14.022699 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:14.022786 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 07:24:14.022832 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:24:14.029316 31967 raft_consensus.cc:1261] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5 [term 1 FOLLOWER]: Refusing update from remote peer d4b13d8f9e2f4824937a761d438a2a7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:14.029359 31781 raft_consensus.cc:1261] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377 [term 1 FOLLOWER]: Refusing update from remote peer d4b13d8f9e2f4824937a761d438a2a7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:14.030107 32255 consensus_queue.cc:1035] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffd56028a9194e7591d28826190ab3c5" member_type: VOTER last_known_addr { host: "127.1.125.194" port: 44369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:14.030310 32257 consensus_queue.cc:1035] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbf4b88ebffa4d869e39e4d1f2d34377" member_type: VOTER last_known_addr { host: "127.1.125.193" port: 39517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:14.038318 32261 mvcc.cc:205] Tried to move back new op lower bound from 7316875894896582656 to 7316875894732414976. Current Snapshot: MvccSnapshot[applied={T|T < 7316875894896582656}]
I20260810 07:24:14.039225 32264 mvcc.cc:205] Tried to move back new op lower bound from 7316875894896582656 to 7316875894732414976. Current Snapshot: MvccSnapshot[applied={T|T < 7316875894896582656}]
I20260810 07:24:14.048034 31557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:24:14.048187 31557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:24:14.050599 31557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abae00fe00084b7ea848063341269ab8: Sending DeleteTablet for 3 replicas of tablet 300bc36a416f4c81aeb8e29a7498ea04
I20260810 07:24:14.051380 32124 tablet_service.cc:1514] Processing DeleteTablet for tablet 300bc36a416f4c81aeb8e29a7498ea04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:14 UTC) from {username='slave'} at 127.0.0.1:35964
I20260810 07:24:14.051388 31753 tablet_service.cc:1514] Processing DeleteTablet for tablet 300bc36a416f4c81aeb8e29a7498ea04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:14 UTC) from {username='slave'} at 127.0.0.1:35920
I20260810 07:24:14.052081 32275 tablet_replica.cc:330] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377: stopping tablet replica
I20260810 07:24:14.052037 31940 tablet_service.cc:1514] Processing DeleteTablet for tablet 300bc36a416f4c81aeb8e29a7498ea04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:24:14 UTC) from {username='slave'} at 127.0.0.1:34762
I20260810 07:24:14.052213 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 31656
I20260810 07:24:14.052434 32275 raft_consensus.cc:2229] T 300bc36a416f4c81aeb8e29a7498ea04 P cbf4b88ebffa4d869e39e4d1f2d34377 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:24:14.053066 32276 tablet_replica.cc:330] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c: stopping tablet replica
I20260810 07:24:14.053296 32276 raft_consensus.cc:2229] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:24:14.053591 32276 raft_consensus.cc:2258] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:24:14.055294 32277 tablet_replica.cc:330] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5: stopping tablet replica
I20260810 07:24:14.055585 32277 raft_consensus.cc:2229] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:24:14.055925 32277 raft_consensus.cc:2258] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:24:14.056550 32276 ts_tablet_manager.cc:1905] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:24:14.060737 32276 ts_tablet_manager.cc:1918] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:24:14.060846 32276 log.cc:1196] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-2/wal/wals/300bc36a416f4c81aeb8e29a7498ea04
I20260810 07:24:14.061280 32276 ts_tablet_manager.cc:1939] T 300bc36a416f4c81aeb8e29a7498ea04 P d4b13d8f9e2f4824937a761d438a2a7c: Deleting consensus metadata
I20260810 07:24:14.063019 31544 catalog_manager.cc:4841] TS d4b13d8f9e2f4824937a761d438a2a7c (127.1.125.195:40341): tablet 300bc36a416f4c81aeb8e29a7498ea04 (table test-table [id=3372fca7584d49f4bac477a2c1f32476]) successfully deleted
I20260810 07:24:14.064028 32277 ts_tablet_manager.cc:1905] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:24:14.067849 32277 ts_tablet_manager.cc:1918] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:24:14.067974 32277 log.cc:1196] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5: Deleting WAL directory at /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346386457763-1527-0/minicluster-data/ts-1/wal/wals/300bc36a416f4c81aeb8e29a7498ea04
I20260810 07:24:14.068338 32277 ts_tablet_manager.cc:1939] T 300bc36a416f4c81aeb8e29a7498ea04 P ffd56028a9194e7591d28826190ab3c5: Deleting consensus metadata
I20260810 07:24:14.069969 31548 catalog_manager.cc:4841] TS ffd56028a9194e7591d28826190ab3c5 (127.1.125.194:44369): tablet 300bc36a416f4c81aeb8e29a7498ea04 (table test-table [id=3372fca7584d49f4bac477a2c1f32476]) successfully deleted
W20260810 07:24:14.072081 31546 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.125.193:39517 (error 108)
I20260810 07:24:14.072197 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 31851
I20260810 07:24:14.079579 32240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:14.079130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.193:39517 (local address 127.0.0.1:35934)
0810 07:24:14.079381 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:14.079478 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.193:39517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260810 07:24:14.079777 31546 catalog_manager.cc:4584] TS cbf4b88ebffa4d869e39e4d1f2d34377 (127.1.125.193:39517): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 300bc36a416f4c81aeb8e29a7498ea04: Network error: Client connection negotiation failed: client connection to 127.1.125.193:39517: connect: Connection refused (error 111)
I20260810 07:24:14.095052 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 32033
I20260810 07:24:14.106213 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 31524
2026-08-10T07:24:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1347 ms)
[----------] 6 tests from EncryptionPolicyTest (11823 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:14 dist-test-slave-44d2 krb5kdc[32286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:14 dist-test-slave-44d2 krb5kdc[32286](info): set up 2 sockets
Aug 10 07:24:14 dist-test-slave-44d2 krb5kdc[32286](info): commencing operation
krb5kdc: starting...
W20260810 07:24:15.381141 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:15 dist-test-slave-44d2 krb5kdc[32286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:15.510846 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43311
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:15.624866 32309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:15.625130 32309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:15.625171 32309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:15.629113 32309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:15.629170 32309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:15.629190 32309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:15.629207 32309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:15.629222 32309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:15.634426 32309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43311
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:15.635499 32309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:15.636555 32309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:15.645051 32318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:15.645180 32316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:15.645277 32315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:15.645687 32309 server_base.cc:1034] running on GCE node
I20260810 07:24:15.646224 32309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:15.647349 32309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:15.648564 32309 hybrid_clock.cc:648] HybridClock initialized: now 1786346655648548 us; error 31 us; skew 500 ppm
Aug 10 07:24:15 dist-test-slave-44d2 krb5kdc[32286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346655, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:15.651381 32309 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:15.652195 32309 webserver.cc:460] Webserver started at http://127.0.0.1:38983/ using document root <none> and password file <none>
I20260810 07:24:15.652786 32309 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:15.652848 32309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:15.653023 32309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:15.655019 32309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "42d5ade22abc43f8a8e6f4cb76a13b63"
format_stamp: "Formatted at 2026-08-10 07:24:15 on dist-test-slave-44d2"
server_key: "7a6b51fe1676af48bbb88109c546bac8"
server_key_iv: "907365480902eedb928836adb9b89907"
server_key_version: "encryptionkey@0"
I20260810 07:24:15.655516 32309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "42d5ade22abc43f8a8e6f4cb76a13b63"
format_stamp: "Formatted at 2026-08-10 07:24:15 on dist-test-slave-44d2"
server_key: "7a6b51fe1676af48bbb88109c546bac8"
server_key_iv: "907365480902eedb928836adb9b89907"
server_key_version: "encryptionkey@0"
I20260810 07:24:15.659219 32309 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:24:15.661585 32325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:15.662783 32309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:24:15.662911 32309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "42d5ade22abc43f8a8e6f4cb76a13b63"
format_stamp: "Formatted at 2026-08-10 07:24:15 on dist-test-slave-44d2"
server_key: "7a6b51fe1676af48bbb88109c546bac8"
server_key_iv: "907365480902eedb928836adb9b89907"
server_key_version: "encryptionkey@0"
I20260810 07:24:15.663035 32309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:15.678056 32309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:15.681353 32309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:15.681578 32309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:15.688766 32309 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43311
I20260810 07:24:15.688876 32401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43311 every 8 connection(s)
I20260810 07:24:15.689719 32309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:15.692449 32402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:15.697459 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 32309
I20260810 07:24:15.697614 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:15.697916 1527 external_mini_cluster.cc:1428] Setting key 50417bd43c5c85629192ab23ef6c90e2
I20260810 07:24:15.698181 32402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63: Bootstrap starting.
I20260810 07:24:15.700729 32402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:15.701733 32402 log.cc:824] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:15.703958 32402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63: No bootstrap required, opened a new log
Aug 10 07:24:15 dist-test-slave-44d2 krb5kdc[32286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:24:15.708634 32402 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d5ade22abc43f8a8e6f4cb76a13b63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43311 } }
I20260810 07:24:15.708918 32402 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:15.708989 32402 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42d5ade22abc43f8a8e6f4cb76a13b63, State: Initialized, Role: FOLLOWER
I20260810 07:24:15.709472 32402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d5ade22abc43f8a8e6f4cb76a13b63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43311 } }
I20260810 07:24:15.709643 32402 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:15.709734 32402 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:15.709836 32402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:15.710718 32402 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d5ade22abc43f8a8e6f4cb76a13b63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43311 } }
I20260810 07:24:15.711066 32402 leader_election.cc:302] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42d5ade22abc43f8a8e6f4cb76a13b63; no voters:
I20260810 07:24:15.711422 32402 leader_election.cc:288] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:15.711485 32408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:15.711707 32408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [term 1 LEADER]: Becoming Leader. State: Replica: 42d5ade22abc43f8a8e6f4cb76a13b63, State: Running, Role: LEADER
I20260810 07:24:15.712008 32408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d5ade22abc43f8a8e6f4cb76a13b63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43311 } }
I20260810 07:24:15.713282 32410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42d5ade22abc43f8a8e6f4cb76a13b63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d5ade22abc43f8a8e6f4cb76a13b63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43311 } } }
I20260810 07:24:15.713500 32410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:15.713536 32406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:15.699234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41193 (local address 127.0.0.1:43311)
0810 07:24:15.699650 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.699676 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.699712 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:24:15.700315 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.700331 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.700735 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.701059 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.702043 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.703142 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.704302 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.704833 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.704953 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.707839 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:15.707844 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:15.707857 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:15.707884 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:15.709988 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.710512 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.710518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.710525 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.710643 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.711035 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.711039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.711040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.711548 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:15.711768 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:15.712160 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 07:24:15.712406 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":144,"threads_started":1}
I20260810 07:24:15.713773 32409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42d5ade22abc43f8a8e6f4cb76a13b63. Latest consensus state: current_term: 1 leader_uuid: "42d5ade22abc43f8a8e6f4cb76a13b63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d5ade22abc43f8a8e6f4cb76a13b63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43311 } } }
I20260810 07:24:15.713874 32409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:15.712795 32402 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:15.714259 32413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:15.716742 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:15.717759 32413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:15.724231 32413 catalog_manager.cc:1347] Generated new cluster ID: 2a4238b1b0e742bba9646ad197470bb6
I20260810 07:24:15.724300 32413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:15.728652 32406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:15.719128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60281 (local address 127.0.0.1:43311)
0810 07:24:15.719240 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:15.719244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:15.719585 (+ 341us) server_negotiation.cc:407] Connection header received
0810 07:24:15.719711 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:15.719715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:15.719761 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:15.719845 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:15.720771 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.721532 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.722747 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:15.723125 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:15.723203 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:15.725079 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:15.725084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:15.725095 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:15.725118 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:15.726774 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.727320 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.727323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.727325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.727406 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:15.727770 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:15.727773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:15.727775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:15.727961 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:15.728094 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:15.728398 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 07:24:15.728506 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 07:24:15.746927 32413 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:15.748276 32413 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:15.754846 32413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42d5ade22abc43f8a8e6f4cb76a13b63: Generated new TSK 0
I20260810 07:24:15.755674 32413 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:15.763652 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 32309
W20260810 07:24:15.775301 32279 connection.cc:535] client connection to 127.0.0.1:43311 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43311): Connection reset by peer (error 104)
2026-08-10T07:24:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1653 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:15 dist-test-slave-44d2 krb5kdc[32439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:15 dist-test-slave-44d2 krb5kdc[32439](info): set up 2 sockets
Aug 10 07:24:15 dist-test-slave-44d2 krb5kdc[32439](info): commencing operation
krb5kdc: starting...
W20260810 07:24:17.020339 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:17 dist-test-slave-44d2 krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:17.155303 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39815
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:17.272625 32463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:17.272889 32463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:17.272931 32463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:17.276916 32463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:17.276975 32463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:17.276995 32463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:17.277012 32463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:17.277027 32463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:17.282256 32463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39815
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:17.283221 32463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:17.284178 32463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:17.292905 32468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:17.293035 32471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:17.293069 32469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:17.293186 32463 server_base.cc:1034] running on GCE node
I20260810 07:24:17.293884 32463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:17.295003 32463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:17.296173 32463 hybrid_clock.cc:648] HybridClock initialized: now 1786346657296146 us; error 43 us; skew 500 ppm
Aug 10 07:24:17 dist-test-slave-44d2 krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346657, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:17.299239 32463 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:17.300164 32463 webserver.cc:460] Webserver started at http://127.0.0.1:38875/ using document root <none> and password file <none>
I20260810 07:24:17.300823 32463 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:17.300925 32463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:17.301165 32463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:17.303217 32463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "796ee33583164f6389945484bf573038"
format_stamp: "Formatted at 2026-08-10 07:24:17 on dist-test-slave-44d2"
server_key: "55cf3dbe9980072b61fd0000ae0e0275"
server_key_iv: "e4478ef3087c234f737d2661a48136a1"
server_key_version: "encryptionkey@0"
I20260810 07:24:17.303787 32463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "796ee33583164f6389945484bf573038"
format_stamp: "Formatted at 2026-08-10 07:24:17 on dist-test-slave-44d2"
server_key: "55cf3dbe9980072b61fd0000ae0e0275"
server_key_iv: "e4478ef3087c234f737d2661a48136a1"
server_key_version: "encryptionkey@0"
I20260810 07:24:17.307758 32463 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 07:24:17.310642 32480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:17.311935 32463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:24:17.312095 32463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "796ee33583164f6389945484bf573038"
format_stamp: "Formatted at 2026-08-10 07:24:17 on dist-test-slave-44d2"
server_key: "55cf3dbe9980072b61fd0000ae0e0275"
server_key_iv: "e4478ef3087c234f737d2661a48136a1"
server_key_version: "encryptionkey@0"
I20260810 07:24:17.312214 32463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:17.331403 32463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:17.334697 32463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:17.334923 32463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:17.342777 32463 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39815
I20260810 07:24:17.342806 32556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39815 every 8 connection(s)
I20260810 07:24:17.343770 32463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:17.346745 32557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:17.352396 32557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038: Bootstrap starting.
I20260810 07:24:17.352990 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 32463
I20260810 07:24:17.353147 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:17.353470 1527 external_mini_cluster.cc:1428] Setting key 7fe51794b3aa2d014bd72a2a8424285f
I20260810 07:24:17.355360 32557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:17.356119 32557 log.cc:824] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:17.358222 32557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038: No bootstrap required, opened a new log
I20260810 07:24:17.362519 32557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796ee33583164f6389945484bf573038" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39815 } }
I20260810 07:24:17.362725 32557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:17.362777 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:17.354828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44397 (local address 127.0.0.1:39815)
0810 07:24:17.355269 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:17.355288 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:17.355325 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:24:17.355994 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:17.356011 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:17.356415 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:17.356844 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:17.357244 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:17.358215 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:17.359443 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:17.359911 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:17.360031 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:17.361006 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:17.361011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:17.361024 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:17.361422 (+ 398us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:17.361475 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:17.361498 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:17.361602 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:17.361838 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 07:24:17.361987 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":151,"threads_started":1}
I20260810 07:24:17.362784 32557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 796ee33583164f6389945484bf573038, State: Initialized, Role: FOLLOWER
I20260810 07:24:17.363353 32557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796ee33583164f6389945484bf573038" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39815 } }
I20260810 07:24:17.363487 32557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:17.363574 32557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:17.363699 32557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:17.364619 32557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796ee33583164f6389945484bf573038" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39815 } }
I20260810 07:24:17.364984 32557 leader_election.cc:302] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 796ee33583164f6389945484bf573038; no voters:
I20260810 07:24:17.365365 32557 leader_election.cc:288] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:17.365465 32562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:17.365705 32562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [term 1 LEADER]: Becoming Leader. State: Replica: 796ee33583164f6389945484bf573038, State: Running, Role: LEADER
I20260810 07:24:17.365947 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:17.366039 32562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796ee33583164f6389945484bf573038" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39815 } }
I20260810 07:24:17.366633 32557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:17.367273 32562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 796ee33583164f6389945484bf573038. Latest consensus state: current_term: 1 leader_uuid: "796ee33583164f6389945484bf573038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796ee33583164f6389945484bf573038" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39815 } } }
I20260810 07:24:17.367412 32562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:17.367729 32564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "796ee33583164f6389945484bf573038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796ee33583164f6389945484bf573038" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39815 } } }
I20260810 07:24:17.367837 32564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038 [sys.catalog]: This master's current role is: LEADER
W20260810 07:24:17.371762 32592 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:17.371857 32592 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:17.371946 32579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:17.372963 32579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:17.373983 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:17.368969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45629 (local address 127.0.0.1:39815)
0810 07:24:17.369265 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:17.369272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:17.369415 (+ 143us) server_negotiation.cc:407] Connection header received
0810 07:24:17.369527 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:17.369530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:17.369571 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:17.369649 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:17.370072 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:17.370812 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:17.371988 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:17.372277 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:17.372358 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:17.373426 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:17.373430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:17.373434 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:17.373447 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:17.373477 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:17.373491 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:17.373591 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:17.373799 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 07:24:17.373836 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209}
I20260810 07:24:17.378355 32579 catalog_manager.cc:1347] Generated new cluster ID: a43ba41c904b450aa6b3cf54eddba1fb
I20260810 07:24:17.378417 32579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:17.436147 32579 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:17.437261 32579 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:17.443899 32579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 796ee33583164f6389945484bf573038: Generated new TSK 0
I20260810 07:24:17.444677 32579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:17.453930 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 32463
W20260810 07:24:17.475279 32437 connection.cc:535] client connection to 127.0.0.1:39815 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39815): Connection reset by peer (error 104)
2026-08-10T07:24:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1700 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:17 dist-test-slave-44d2 krb5kdc[32607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:17 dist-test-slave-44d2 krb5kdc[32607](info): set up 2 sockets
Aug 10 07:24:17 dist-test-slave-44d2 krb5kdc[32607](info): commencing operation
krb5kdc: starting...
W20260810 07:24:18.722203 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:18 dist-test-slave-44d2 krb5kdc[32607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:18.853333 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:18.971164 32627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:18.971431 32627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:18.971513 32627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:18.975597 32627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:18.975664 32627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:18.975709 32627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:18.975754 32627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:18.975790 32627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:18.981240 32627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44823
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:18.982301 32627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:18.983353 32627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:24:18.992426 32627 server_base.cc:1034] running on GCE node
W20260810 07:24:18.992395 32638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:18.992359 32641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:18.992372 32639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:18.993299 32627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:18.994366 32627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:18.995600 32627 hybrid_clock.cc:648] HybridClock initialized: now 1786346658995582 us; error 31 us; skew 500 ppm
Aug 10 07:24:18 dist-test-slave-44d2 krb5kdc[32607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346658, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:19.002010 32627 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:19.002951 32627 webserver.cc:460] Webserver started at http://127.0.0.1:33461/ using document root <none> and password file <none>
I20260810 07:24:19.003490 32627 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:19.003561 32627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:19.003731 32627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:19.005824 32627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "5e05700c502b413eb422ddce1530fe40"
format_stamp: "Formatted at 2026-08-10 07:24:18 on dist-test-slave-44d2"
server_key: "cc4bf3c9ec12c5d4d87699c699dcb4fa"
server_key_iv: "71060aff4ad532a4d18e8600159f463e"
server_key_version: "encryptionkey@0"
I20260810 07:24:19.006335 32627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "5e05700c502b413eb422ddce1530fe40"
format_stamp: "Formatted at 2026-08-10 07:24:18 on dist-test-slave-44d2"
server_key: "cc4bf3c9ec12c5d4d87699c699dcb4fa"
server_key_iv: "71060aff4ad532a4d18e8600159f463e"
server_key_version: "encryptionkey@0"
I20260810 07:24:19.010097 32627 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:24:19.012740 32651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:19.014101 32627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:24:19.014204 32627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "5e05700c502b413eb422ddce1530fe40"
format_stamp: "Formatted at 2026-08-10 07:24:18 on dist-test-slave-44d2"
server_key: "cc4bf3c9ec12c5d4d87699c699dcb4fa"
server_key_iv: "71060aff4ad532a4d18e8600159f463e"
server_key_version: "encryptionkey@0"
I20260810 07:24:19.014288 32627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:19.018833 32627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:19.021656 32627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:19.021795 32627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:19.029058 32627 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44823
I20260810 07:24:19.029081 32732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44823 every 8 connection(s)
I20260810 07:24:19.030031 32627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:19.032867 32733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:19.038360 32733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40: Bootstrap starting.
I20260810 07:24:19.039815 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 32627
I20260810 07:24:19.039966 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:19.040308 1527 external_mini_cluster.cc:1428] Setting key e661d9e3c638effef25cb3ecb3f69ed0
I20260810 07:24:19.041112 32733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:19.042026 32733 log.cc:824] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:19.044121 32733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40: No bootstrap required, opened a new log
I20260810 07:24:19.045758 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:19.041485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53929 (local address 127.0.0.1:44823)
0810 07:24:19.041928 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:19.041942 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:19.041977 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:24:19.042665 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:19.042683 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:19.042956 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:19.043301 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:19.044289 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:19.044294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:19.044332 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:19.044352 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:19.044449 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:19.044529 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:19.044676 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:19.044822 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 07:24:19.045013 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":124,"threads_started":1}
I20260810 07:24:19.049149 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:19.049136 32733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e05700c502b413eb422ddce1530fe40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44823 } }
I20260810 07:24:19.049443 32733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:19.049530 32733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e05700c502b413eb422ddce1530fe40, State: Initialized, Role: FOLLOWER
I20260810 07:24:19.050009 32733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e05700c502b413eb422ddce1530fe40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44823 } }
I20260810 07:24:19.050174 32733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:19.050271 32733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:19.050396 32733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:19.051623 32733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e05700c502b413eb422ddce1530fe40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44823 } }
I20260810 07:24:19.051985 32733 leader_election.cc:302] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e05700c502b413eb422ddce1530fe40; no voters:
I20260810 07:24:19.052342 32733 leader_election.cc:288] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:19.052409 32748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:19.052743 32748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [term 1 LEADER]: Becoming Leader. State: Replica: 5e05700c502b413eb422ddce1530fe40, State: Running, Role: LEADER
I20260810 07:24:19.053328 32748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e05700c502b413eb422ddce1530fe40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44823 } }
I20260810 07:24:19.053682 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:19.051453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52323 (local address 127.0.0.1:44823)
0810 07:24:19.051590 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:19.051594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:19.051811 (+ 217us) server_negotiation.cc:407] Connection header received
0810 07:24:19.051947 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:19.051950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:19.051983 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:19.052079 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:19.052941 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:19.052944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:19.052948 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:19.052963 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:19.052991 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:19.053004 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:19.053084 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:19.053501 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 07:24:19.053545 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 07:24:19.053901 32733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:19.055061 32749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e05700c502b413eb422ddce1530fe40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e05700c502b413eb422ddce1530fe40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44823 } } }
I20260810 07:24:19.055200 32749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:19.055608 32762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:19.055578 32750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e05700c502b413eb422ddce1530fe40. Latest consensus state: current_term: 1 leader_uuid: "5e05700c502b413eb422ddce1530fe40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e05700c502b413eb422ddce1530fe40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44823 } } }
I20260810 07:24:19.055720 32750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:19.058949 32762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:19.064327 32762 catalog_manager.cc:1347] Generated new cluster ID: 4a453e74b683488d804b230e25ef3d9d
I20260810 07:24:19.064400 32762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:19.071283 32762 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:19.071578 32762 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:19.075477 32762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e05700c502b413eb422ddce1530fe40: Generated new TSK 0
I20260810 07:24:19.076077 32762 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:19.081007 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 32627
2026-08-10T07:24:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1618 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:19 dist-test-slave-44d2 krb5kdc[314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:19 dist-test-slave-44d2 krb5kdc[314](info): set up 2 sockets
Aug 10 07:24:19 dist-test-slave-44d2 krb5kdc[314](info): commencing operation
krb5kdc: starting...
W20260810 07:24:20.383808 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:20 dist-test-slave-44d2 krb5kdc[314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:20.515228 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35903
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:20.629144 330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:20.629433 330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:20.629518 330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:20.633424 330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:20.633512 330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:20.633553 330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:20.633611 330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:20.633652 330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:20.638688 330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:20.639729 330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:20.640751 330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:20.650637 337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:20.650614 343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:20.650718 339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:20.650765 330 server_base.cc:1034] running on GCE node
I20260810 07:24:20.651590 330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:20.653025 330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:20.654230 330 hybrid_clock.cc:648] HybridClock initialized: now 1786346660654204 us; error 40 us; skew 500 ppm
Aug 10 07:24:20 dist-test-slave-44d2 krb5kdc[314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346660, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:20.657595 330 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:20.658490 330 webserver.cc:460] Webserver started at http://127.0.0.1:40687/ using document root <none> and password file <none>
I20260810 07:24:20.659068 330 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:20.659164 330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:20.659399 330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:20.661514 330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "961f57999942415b87cef6b86782d07d"
format_stamp: "Formatted at 2026-08-10 07:24:20 on dist-test-slave-44d2"
server_key: "5bbe7cf900aeeee3a725482cc7eeb521"
server_key_iv: "95f0778be1867645ed63cb0d1d374439"
server_key_version: "encryptionkey@0"
I20260810 07:24:20.662098 330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "961f57999942415b87cef6b86782d07d"
format_stamp: "Formatted at 2026-08-10 07:24:20 on dist-test-slave-44d2"
server_key: "5bbe7cf900aeeee3a725482cc7eeb521"
server_key_iv: "95f0778be1867645ed63cb0d1d374439"
server_key_version: "encryptionkey@0"
I20260810 07:24:20.665828 330 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:24:20.668350 356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:20.669596 330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:24:20.669757 330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "961f57999942415b87cef6b86782d07d"
format_stamp: "Formatted at 2026-08-10 07:24:20 on dist-test-slave-44d2"
server_key: "5bbe7cf900aeeee3a725482cc7eeb521"
server_key_iv: "95f0778be1867645ed63cb0d1d374439"
server_key_version: "encryptionkey@0"
I20260810 07:24:20.669868 330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:20.701749 330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:20.704773 330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:20.705001 330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:20.712162 330 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35903
I20260810 07:24:20.712188 428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35903 every 8 connection(s)
I20260810 07:24:20.713248 330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:20.716171 430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:20.721871 430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d: Bootstrap starting.
I20260810 07:24:20.722627 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 330
I20260810 07:24:20.722779 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:20.723084 1527 external_mini_cluster.cc:1428] Setting key 719456d32a84c4c98d0f6206edc49f0b
I20260810 07:24:20.724857 430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:20.725760 430 log.cc:824] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:20.727636 430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d: No bootstrap required, opened a new log
I20260810 07:24:20.731045 430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "961f57999942415b87cef6b86782d07d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35903 } }
I20260810 07:24:20.731285 430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:20.731377 430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 961f57999942415b87cef6b86782d07d, State: Initialized, Role: FOLLOWER
I20260810 07:24:20.731848 430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "961f57999942415b87cef6b86782d07d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35903 } }
I20260810 07:24:20.732009 430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:20.732098 430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:20.732198 430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [term 0 FOLLOWER]: Advancing to term 1
Aug 10 07:24:20 dist-test-slave-44d2 krb5kdc[314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:24:20.733152 430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "961f57999942415b87cef6b86782d07d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35903 } }
I20260810 07:24:20.733542 430 leader_election.cc:302] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 961f57999942415b87cef6b86782d07d; no voters:
I20260810 07:24:20.733922 430 leader_election.cc:288] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:20.734040 435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:20.734305 435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [term 1 LEADER]: Becoming Leader. State: Replica: 961f57999942415b87cef6b86782d07d, State: Running, Role: LEADER
I20260810 07:24:20.734717 435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "961f57999942415b87cef6b86782d07d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35903 } }
I20260810 07:24:20.735270 430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:20.736608 436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "961f57999942415b87cef6b86782d07d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "961f57999942415b87cef6b86782d07d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35903 } } }
I20260810 07:24:20.736742 436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:20.737044 437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 961f57999942415b87cef6b86782d07d. Latest consensus state: current_term: 1 leader_uuid: "961f57999942415b87cef6b86782d07d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "961f57999942415b87cef6b86782d07d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35903 } } }
I20260810 07:24:20.737149 437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:20.739429 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:20.724318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45747 (local address 127.0.0.1:35903)
0810 07:24:20.724835 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:20.724856 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:20.724903 (+ 47us) server_negotiation.cc:407] Connection header received
0810 07:24:20.725769 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:20.725794 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:20.726243 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:20.726587 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:20.727431 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:20.728531 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:20.729425 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:20.729931 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:20.730140 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:20.733055 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:20.733062 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:20.733090 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:20.733119 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:20.735426 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:20.736009 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:20.736017 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:20.736024 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:20.736187 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:20.736610 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:20.736614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:20.736619 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:20.737192 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:20.737432 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:20.738449 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0810 07:24:20.738717 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":147,"threads_started":1}
W20260810 07:24:20.740609 452 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:20.740737 452 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:20.740813 446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:20.742025 446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:20.742460 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:20.750540 446 catalog_manager.cc:1347] Generated new cluster ID: aad7993f52904a1db50fe67daf0bdf01
I20260810 07:24:20.750599 446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:20.754918 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:20.744973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54651 (local address 127.0.0.1:35903)
0810 07:24:20.745085 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:20.745088 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:20.745100 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:24:20.745150 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:20.745152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:20.745190 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:20.745279 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:20.745935 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:20.746504 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:20.747694 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:20.748049 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:20.748161 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:20.749606 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:20.749611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:20.749613 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:20.749645 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:20.753193 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:20.753678 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:20.753682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:20.753683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:20.753765 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:20.754075 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:20.754077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:20.754079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:20.754288 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:20.754401 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:20.754665 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 07:24:20.754770 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":44}
I20260810 07:24:20.761471 446 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:20.762706 446 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:20.771026 446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 961f57999942415b87cef6b86782d07d: Generated new TSK 0
I20260810 07:24:20.771719 446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:20.778602 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 330
2026-08-10T07:24:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1700 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:20 dist-test-slave-44d2 krb5kdc[468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:20 dist-test-slave-44d2 krb5kdc[468](info): set up 2 sockets
Aug 10 07:24:20 dist-test-slave-44d2 krb5kdc[468](info): commencing operation
krb5kdc: starting...
W20260810 07:24:22.038985 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:22 dist-test-slave-44d2 krb5kdc[468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:22.189826 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44289
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:22.316857 487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:22.317176 487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:22.317260 487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:22.321414 487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:22.321507 487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:22.321552 487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:22.321614 487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:22.321656 487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:22.327116 487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:22.328248 487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:22.329461 487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:22.338407 498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:22.338420 508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:22.338464 487 server_base.cc:1034] running on GCE node
W20260810 07:24:22.338474 497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:22.339277 487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:22.340380 487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:22.341552 487 hybrid_clock.cc:648] HybridClock initialized: now 1786346662341537 us; error 27 us; skew 500 ppm
Aug 10 07:24:22 dist-test-slave-44d2 krb5kdc[468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346662, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:22.344328 487 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:22.345234 487 webserver.cc:460] Webserver started at http://127.0.0.1:38503/ using document root <none> and password file <none>
I20260810 07:24:22.345770 487 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:22.345834 487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:22.346014 487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:22.348099 487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "9c42144dce04426db869d15aef2fbdfb"
format_stamp: "Formatted at 2026-08-10 07:24:22 on dist-test-slave-44d2"
server_key: "c904c0f52c2f3f72b0a1d850b7ee636f"
server_key_iv: "0b6e66a195b59af84b002bc1af92c076"
server_key_version: "encryptionkey@0"
I20260810 07:24:22.348682 487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "9c42144dce04426db869d15aef2fbdfb"
format_stamp: "Formatted at 2026-08-10 07:24:22 on dist-test-slave-44d2"
server_key: "c904c0f52c2f3f72b0a1d850b7ee636f"
server_key_iv: "0b6e66a195b59af84b002bc1af92c076"
server_key_version: "encryptionkey@0"
I20260810 07:24:22.352376 487 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:24:22.354813 517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:22.356036 487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:24:22.356168 487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "9c42144dce04426db869d15aef2fbdfb"
format_stamp: "Formatted at 2026-08-10 07:24:22 on dist-test-slave-44d2"
server_key: "c904c0f52c2f3f72b0a1d850b7ee636f"
server_key_iv: "0b6e66a195b59af84b002bc1af92c076"
server_key_version: "encryptionkey@0"
I20260810 07:24:22.356271 487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:22.371744 487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:22.374601 487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:22.374756 487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:22.381857 487 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44289
I20260810 07:24:22.381946 593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44289 every 8 connection(s)
I20260810 07:24:22.382834 487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:22.385756 594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:22.387135 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 487
I20260810 07:24:22.387284 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:22.387596 1527 external_mini_cluster.cc:1428] Setting key e32eeadf060515589a8bf27a9dc44945
I20260810 07:24:22.392819 594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb: Bootstrap starting.
I20260810 07:24:22.395484 594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:22.396431 594 log.cc:824] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:22.397902 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:22.389064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38887 (local address 127.0.0.1:44289)
0810 07:24:22.389509 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:22.389544 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:22.389598 (+ 54us) server_negotiation.cc:407] Connection header received
0810 07:24:22.390430 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:22.390455 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:22.390914 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:22.391288 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:22.391759 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:22.393292 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:22.394225 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:22.394914 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:22.395077 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:22.395928 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:22.395935 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:22.395952 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:22.396459 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:22.396563 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:22.396610 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:22.396704 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:22.396867 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 07:24:22.397088 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":104,"threads_started":1}
I20260810 07:24:22.398717 594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb: No bootstrap required, opened a new log
I20260810 07:24:22.400830 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:22.402740 594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c42144dce04426db869d15aef2fbdfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44289 } }
I20260810 07:24:22.403103 594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:22.403183 594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c42144dce04426db869d15aef2fbdfb, State: Initialized, Role: FOLLOWER
I20260810 07:24:22.403685 594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c42144dce04426db869d15aef2fbdfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44289 } }
I20260810 07:24:22.403879 594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:22.403949 594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:22.404055 594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:22.405079 594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c42144dce04426db869d15aef2fbdfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44289 } }
I20260810 07:24:22.405468 594 leader_election.cc:302] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c42144dce04426db869d15aef2fbdfb; no voters:
I20260810 07:24:22.405817 594 leader_election.cc:288] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:22.405941 614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:22.406172 614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [term 1 LEADER]: Becoming Leader. State: Replica: 9c42144dce04426db869d15aef2fbdfb, State: Running, Role: LEADER
I20260810 07:24:22.406554 614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c42144dce04426db869d15aef2fbdfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44289 } }
I20260810 07:24:22.407368 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:22.402910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48127 (local address 127.0.0.1:44289)
0810 07:24:22.403030 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:22.403035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:22.403205 (+ 170us) server_negotiation.cc:407] Connection header received
0810 07:24:22.403353 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:22.403356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:22.403402 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:22.403495 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:22.403771 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:22.404529 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:22.405341 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:22.405728 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:22.405874 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:22.406777 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:22.406781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:22.406784 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:22.406797 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:22.406826 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:22.406839 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:22.406911 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:22.407194 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 07:24:22.407250 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:24:22.407588 594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:22.408356 616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c42144dce04426db869d15aef2fbdfb. Latest consensus state: current_term: 1 leader_uuid: "9c42144dce04426db869d15aef2fbdfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c42144dce04426db869d15aef2fbdfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44289 } } }
I20260810 07:24:22.408597 616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:22.408670 615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c42144dce04426db869d15aef2fbdfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c42144dce04426db869d15aef2fbdfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44289 } } }
I20260810 07:24:22.408854 615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:22.409904 627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:22.412690 627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:22.417701 627 catalog_manager.cc:1347] Generated new cluster ID: 0688488fa3e341ad85c737605b298d13
I20260810 07:24:22.417766 627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:22.429273 627 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:22.430178 627 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:22.436710 627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c42144dce04426db869d15aef2fbdfb: Generated new TSK 0
I20260810 07:24:22.437350 627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:22.446643 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 487
2026-08-10T07:24:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1664 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:22 dist-test-slave-44d2 krb5kdc[641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:22 dist-test-slave-44d2 krb5kdc[641](info): set up 2 sockets
Aug 10 07:24:22 dist-test-slave-44d2 krb5kdc[641](info): commencing operation
krb5kdc: starting...
W20260810 07:24:23.744335 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:23 dist-test-slave-44d2 krb5kdc[641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:23.875661 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38727
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:23.994237 661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:23.994578 661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:23.994674 661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:23.998734 661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:23.998849 661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:23.998903 661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:23.998967 661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:23.999012 661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:24.004565 661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:24.005656 661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:24.006736 661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:24.015589 674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:24.015666 661 server_base.cc:1034] running on GCE node
W20260810 07:24:24.015614 672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:24.015614 676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:24.016407 661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:24.017557 661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:24.018756 661 hybrid_clock.cc:648] HybridClock initialized: now 1786346664018731 us; error 42 us; skew 500 ppm
Aug 10 07:24:24 dist-test-slave-44d2 krb5kdc[641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346664, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:24.026340 661 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:24.027261 661 webserver.cc:460] Webserver started at http://127.0.0.1:39511/ using document root <none> and password file <none>
I20260810 07:24:24.027789 661 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:24.027850 661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:24.028021 661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:24.030215 661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "c49d6249f1c248a0ac4151e4229bb158"
format_stamp: "Formatted at 2026-08-10 07:24:24 on dist-test-slave-44d2"
server_key: "09f882b6e47bd887cbeacdd45619da85"
server_key_iv: "1d2a1c0974c2dd8407d274d6b341c2c8"
server_key_version: "encryptionkey@0"
I20260810 07:24:24.030737 661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "c49d6249f1c248a0ac4151e4229bb158"
format_stamp: "Formatted at 2026-08-10 07:24:24 on dist-test-slave-44d2"
server_key: "09f882b6e47bd887cbeacdd45619da85"
server_key_iv: "1d2a1c0974c2dd8407d274d6b341c2c8"
server_key_version: "encryptionkey@0"
I20260810 07:24:24.034557 661 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260810 07:24:24.037371 687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:24.038638 661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:24:24.038751 661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "c49d6249f1c248a0ac4151e4229bb158"
format_stamp: "Formatted at 2026-08-10 07:24:24 on dist-test-slave-44d2"
server_key: "09f882b6e47bd887cbeacdd45619da85"
server_key_iv: "1d2a1c0974c2dd8407d274d6b341c2c8"
server_key_version: "encryptionkey@0"
I20260810 07:24:24.038908 661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:24.043903 661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:24.047322 661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:24.047541 661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:24.055466 661 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38727
I20260810 07:24:24.055476 761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38727 every 8 connection(s)
I20260810 07:24:24.056420 661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:24.059322 762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:24.062492 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 661
I20260810 07:24:24.062660 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:24.062974 1527 external_mini_cluster.cc:1428] Setting key 23d2a89cce51f2ade1c0e7fe7c33f0af
I20260810 07:24:24.065336 762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158: Bootstrap starting.
I20260810 07:24:24.067914 762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:24.068178 766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:24.064327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57193 (local address 127.0.0.1:38727)
0810 07:24:24.064793 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:24.064809 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:24.064845 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:24:24.065521 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:24.065540 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:24.065826 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:24.066108 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:24.066939 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:24.066944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:24.066956 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:24.066967 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:24.067020 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:24.067039 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:24.067155 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:24.067252 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 07:24:24.067422 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":189,"threads_started":1}
I20260810 07:24:24.068799 762 log.cc:824] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:24.070946 762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158: No bootstrap required, opened a new log
I20260810 07:24:24.071288 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:24.075222 766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:24.073589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58043 (local address 127.0.0.1:38727)
0810 07:24:24.073719 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:24.073724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:24.073931 (+ 207us) server_negotiation.cc:407] Connection header received
0810 07:24:24.074015 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:24.074018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:24.074052 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:24.074137 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:24.074847 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:24.074852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:24.074857 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:24.074870 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:24.074897 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:24.074909 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:24.075004 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:24.075067 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 07:24:24.075106 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:24:24.075670 762 raft_consensus.cc:348] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c49d6249f1c248a0ac4151e4229bb158" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38727 } }
I20260810 07:24:24.075950 762 raft_consensus.cc:374] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:24.076009 762 raft_consensus.cc:729] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c49d6249f1c248a0ac4151e4229bb158, State: Initialized, Role: FOLLOWER
I20260810 07:24:24.076409 762 consensus_queue.cc:260] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c49d6249f1c248a0ac4151e4229bb158" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38727 } }
I20260810 07:24:24.076601 762 raft_consensus.cc:388] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:24.076653 762 raft_consensus.cc:482] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:24.076725 762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:24.077591 762 raft_consensus.cc:504] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c49d6249f1c248a0ac4151e4229bb158" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38727 } }
I20260810 07:24:24.077881 762 leader_election.cc:302] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c49d6249f1c248a0ac4151e4229bb158; no voters:
I20260810 07:24:24.078151 762 leader_election.cc:288] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:24.078277 777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:24.078518 777 raft_consensus.cc:686] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [term 1 LEADER]: Becoming Leader. State: Replica: c49d6249f1c248a0ac4151e4229bb158, State: Running, Role: LEADER
I20260810 07:24:24.078860 777 consensus_queue.cc:237] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c49d6249f1c248a0ac4151e4229bb158" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38727 } }
I20260810 07:24:24.079371 762 sys_catalog.cc:564] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:24.080612 780 sys_catalog.cc:455] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c49d6249f1c248a0ac4151e4229bb158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c49d6249f1c248a0ac4151e4229bb158" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38727 } } }
I20260810 07:24:24.080737 780 sys_catalog.cc:458] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:24.081164 788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:24.081146 781 sys_catalog.cc:455] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c49d6249f1c248a0ac4151e4229bb158. Latest consensus state: current_term: 1 leader_uuid: "c49d6249f1c248a0ac4151e4229bb158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c49d6249f1c248a0ac4151e4229bb158" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38727 } } }
I20260810 07:24:24.081930 781 sys_catalog.cc:458] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:24.084311 788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:24.089334 788 catalog_manager.cc:1347] Generated new cluster ID: e18e77ca10a741c2b5cdbbe28c7a83cd
I20260810 07:24:24.089398 788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:24.102015 788 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:24.102361 788 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:24.114168 788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c49d6249f1c248a0ac4151e4229bb158: Generated new TSK 0
I20260810 07:24:24.115018 788 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:24.122709 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 661
2026-08-10T07:24:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1687 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:24 dist-test-slave-44d2 krb5kdc[805](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:24 dist-test-slave-44d2 krb5kdc[805](info): set up 2 sockets
Aug 10 07:24:24 dist-test-slave-44d2 krb5kdc[805](info): commencing operation
krb5kdc: starting...
W20260810 07:24:25.412993 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:25 dist-test-slave-44d2 krb5kdc[805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:25.545754 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40883
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:25.663177 824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:25.663441 824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:25.663482 824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:25.667480 824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:25.667546 824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:25.667565 824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:25.667583 824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:25.667598 824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:25.672960 824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:25.674057 824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:25.675159 824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:25.684307 831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:25.684315 834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:25.684355 830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:25.684553 824 server_base.cc:1034] running on GCE node
I20260810 07:24:25.685304 824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:25.686455 824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:25.687659 824 hybrid_clock.cc:648] HybridClock initialized: now 1786346665687631 us; error 40 us; skew 500 ppm
Aug 10 07:24:25 dist-test-slave-44d2 krb5kdc[805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346665, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:25.690500 824 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:25.691330 824 webserver.cc:460] Webserver started at http://127.0.0.1:43091/ using document root <none> and password file <none>
I20260810 07:24:25.691864 824 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:25.691923 824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:25.692144 824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:25.694239 824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "200de51a733d45c7965222691d9548ca"
format_stamp: "Formatted at 2026-08-10 07:24:25 on dist-test-slave-44d2"
server_key: "f266fd1599681939c6b531ff96f7735c"
server_key_iv: "76d57fea7c4b06e64bc1cbd43380ce46"
server_key_version: "encryptionkey@0"
I20260810 07:24:25.694759 824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "200de51a733d45c7965222691d9548ca"
format_stamp: "Formatted at 2026-08-10 07:24:25 on dist-test-slave-44d2"
server_key: "f266fd1599681939c6b531ff96f7735c"
server_key_iv: "76d57fea7c4b06e64bc1cbd43380ce46"
server_key_version: "encryptionkey@0"
I20260810 07:24:25.698426 824 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260810 07:24:25.700836 844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:25.702075 824 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260810 07:24:25.702178 824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "200de51a733d45c7965222691d9548ca"
format_stamp: "Formatted at 2026-08-10 07:24:25 on dist-test-slave-44d2"
server_key: "f266fd1599681939c6b531ff96f7735c"
server_key_iv: "76d57fea7c4b06e64bc1cbd43380ce46"
server_key_version: "encryptionkey@0"
I20260810 07:24:25.702328 824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:25.718369 824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:25.721180 824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:25.721338 824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:25.728619 824 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40883
I20260810 07:24:25.728698 920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40883 every 8 connection(s)
I20260810 07:24:25.729622 824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:25.732621 921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:25.732960 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 824
I20260810 07:24:25.733142 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:25.733450 1527 external_mini_cluster.cc:1428] Setting key d84cd73fb3423313ec9f1bd5bcdd5976
I20260810 07:24:25.739341 921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca: Bootstrap starting.
I20260810 07:24:25.741811 921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca: Neither blocks nor log segments found. Creating new log.
Aug 10 07:24:25 dist-test-slave-44d2 krb5kdc[805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:24:25.742794 921 log.cc:824] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:25.745061 921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca: No bootstrap required, opened a new log
I20260810 07:24:25.748652 924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:25.734966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:57781 (local address 127.0.0.1:40883)
0810 07:24:25.735427 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:25.735442 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:25.735477 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:24:25.736160 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:25.736177 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:25.736588 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:25.736926 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:25.737796 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:25.738770 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:25.739721 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:25.740187 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:25.740356 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:25.743247 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:25.743256 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:25.743274 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:25.743310 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:25.745227 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:25.745797 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:25.745802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:25.745807 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:25.745913 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:25.746330 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:25.746333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:25.746335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:25.746812 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:25.747014 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:25.747456 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 07:24:25.747747 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":125,"threads_started":1}
I20260810 07:24:25.749218 921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "200de51a733d45c7965222691d9548ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40883 } }
I20260810 07:24:25.749433 921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:25.749493 921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 200de51a733d45c7965222691d9548ca, State: Initialized, Role: FOLLOWER
I20260810 07:24:25.749969 921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "200de51a733d45c7965222691d9548ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40883 } }
I20260810 07:24:25.750133 921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:25.750223 921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:25.750350 921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:25.751389 921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "200de51a733d45c7965222691d9548ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40883 } }
I20260810 07:24:25.751777 921 leader_election.cc:302] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 200de51a733d45c7965222691d9548ca; no voters:
I20260810 07:24:25.752166 921 leader_election.cc:288] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:25.752287 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:25.752290 926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:25.752616 926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [term 1 LEADER]: Becoming Leader. State: Replica: 200de51a733d45c7965222691d9548ca, State: Running, Role: LEADER
I20260810 07:24:25.753022 926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "200de51a733d45c7965222691d9548ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40883 } }
I20260810 07:24:25.753615 921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:25.754472 926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 200de51a733d45c7965222691d9548ca. Latest consensus state: current_term: 1 leader_uuid: "200de51a733d45c7965222691d9548ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "200de51a733d45c7965222691d9548ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40883 } } }
I20260810 07:24:25.754704 926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:25.754984 927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "200de51a733d45c7965222691d9548ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "200de51a733d45c7965222691d9548ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40883 } } }
I20260810 07:24:25.755108 927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:25.755326 950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:25.758989 950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:25.765787 924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:25.754637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:40149 (local address 127.0.0.1:40883)
0810 07:24:25.754757 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:25.754762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:25.756784 (+ 2022us) server_negotiation.cc:407] Connection header received
0810 07:24:25.756988 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:25.756991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:25.757058 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:25.757166 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:25.758008 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:25.758826 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:25.759889 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:25.760274 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:25.760435 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:25.761880 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:25.761884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:25.761888 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:25.761917 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:25.763383 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:25.763890 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:25.763899 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:25.763901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:25.763978 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:25.764437 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:25.764442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:25.764444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:25.764752 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:25.764954 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:25.765265 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 07:24:25.765386 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:24:25.765805 950 catalog_manager.cc:1347] Generated new cluster ID: a8a88bb370a64fa490203ece472a5159
I20260810 07:24:25.766299 950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:25.792068 950 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:25.793159 950 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:25.799168 950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 200de51a733d45c7965222691d9548ca: Generated new TSK 0
I20260810 07:24:25.799852 950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:25.804874 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 824
W20260810 07:24:25.819357 803 connection.cc:535] client connection to 127.0.0.1:40883 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40883): Connection reset by peer (error 104)
2026-08-10T07:24:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1674 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:25 dist-test-slave-44d2 krb5kdc[965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:25 dist-test-slave-44d2 krb5kdc[965](info): set up 2 sockets
Aug 10 07:24:25 dist-test-slave-44d2 krb5kdc[965](info): commencing operation
krb5kdc: starting...
W20260810 07:24:27.062348 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:27 dist-test-slave-44d2 krb5kdc[965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:27.196152 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:27.324272 983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:27.324654 983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:27.324744 983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:27.328889 983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:27.328977 983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:27.329030 983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:27.329070 983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:27.329113 983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:27.334525 983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:27.335611 983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:27.336757 983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:27.345384 990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:27.345381 989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:27.345377 992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:27.345685 983 server_base.cc:1034] running on GCE node
I20260810 07:24:27.346302 983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:27.347501 983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:27.348757 983 hybrid_clock.cc:648] HybridClock initialized: now 1786346667348723 us; error 42 us; skew 500 ppm
Aug 10 07:24:27 dist-test-slave-44d2 krb5kdc[965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346667, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:27.351891 983 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:27.352910 983 webserver.cc:460] Webserver started at http://127.0.0.1:44699/ using document root <none> and password file <none>
I20260810 07:24:27.353518 983 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:27.353615 983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:27.353863 983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:27.355971 983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "8d259b15264543f9bc5b74f7476273cb"
format_stamp: "Formatted at 2026-08-10 07:24:27 on dist-test-slave-44d2"
server_key: "7acbbe186a4f2d6d34c0eddf9d8382a5"
server_key_iv: "483a40d71e6d2213538c681704adde7c"
server_key_version: "encryptionkey@0"
I20260810 07:24:27.356627 983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "8d259b15264543f9bc5b74f7476273cb"
format_stamp: "Formatted at 2026-08-10 07:24:27 on dist-test-slave-44d2"
server_key: "7acbbe186a4f2d6d34c0eddf9d8382a5"
server_key_iv: "483a40d71e6d2213538c681704adde7c"
server_key_version: "encryptionkey@0"
I20260810 07:24:27.360421 983 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:24:27.362862 1002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:27.364158 983 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:24:27.364305 983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "8d259b15264543f9bc5b74f7476273cb"
format_stamp: "Formatted at 2026-08-10 07:24:27 on dist-test-slave-44d2"
server_key: "7acbbe186a4f2d6d34c0eddf9d8382a5"
server_key_iv: "483a40d71e6d2213538c681704adde7c"
server_key_version: "encryptionkey@0"
I20260810 07:24:27.364419 983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:27.374161 983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:27.377405 983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:27.377607 983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:27.384591 983 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33961
I20260810 07:24:27.384712 1091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33961 every 8 connection(s)
I20260810 07:24:27.385560 983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:27.388330 1093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:27.393838 1093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb: Bootstrap starting.
I20260810 07:24:27.394137 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 983
I20260810 07:24:27.394284 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:27.394584 1527 external_mini_cluster.cc:1428] Setting key 50e19432406507471eeac7f5b7a9a88f
I20260810 07:24:27.396618 1093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:27.397511 1093 log.cc:824] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:27.399498 1093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb: No bootstrap required, opened a new log
I20260810 07:24:27.404006 1093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d259b15264543f9bc5b74f7476273cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33961 } }
I20260810 07:24:27.404285 1093 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:27.404345 1093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d259b15264543f9bc5b74f7476273cb, State: Initialized, Role: FOLLOWER
I20260810 07:24:27.404652 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:27.395988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:58377 (local address 127.0.0.1:33961)
0810 07:24:27.396420 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:27.396443 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:27.396517 (+ 74us) server_negotiation.cc:407] Connection header received
0810 07:24:27.397362 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:27.397388 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:27.397848 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:27.398232 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:27.398664 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:27.399649 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:27.400634 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:27.401376 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:27.401583 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:27.402479 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:27.402489 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:27.402507 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:27.403068 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:27.403143 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:27.403177 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:27.403319 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:27.403499 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 07:24:27.403649 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":117,"threads_started":1}
I20260810 07:24:27.404860 1093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d259b15264543f9bc5b74f7476273cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33961 } }
I20260810 07:24:27.405050 1093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:27.405149 1093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:27.405253 1093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:27.406112 1093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d259b15264543f9bc5b74f7476273cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33961 } }
I20260810 07:24:27.406495 1093 leader_election.cc:302] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d259b15264543f9bc5b74f7476273cb; no voters:
I20260810 07:24:27.406833 1093 leader_election.cc:288] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:27.406951 1099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:27.407218 1099 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [term 1 LEADER]: Becoming Leader. State: Replica: 8d259b15264543f9bc5b74f7476273cb, State: Running, Role: LEADER
I20260810 07:24:27.407637 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:27.407564 1099 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d259b15264543f9bc5b74f7476273cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33961 } }
I20260810 07:24:27.408133 1093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:27.409952 1100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d259b15264543f9bc5b74f7476273cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d259b15264543f9bc5b74f7476273cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33961 } } }
I20260810 07:24:27.410105 1100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:27.410584 1101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d259b15264543f9bc5b74f7476273cb. Latest consensus state: current_term: 1 leader_uuid: "8d259b15264543f9bc5b74f7476273cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d259b15264543f9bc5b74f7476273cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33961 } } }
I20260810 07:24:27.410938 1101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb [sys.catalog]: This master's current role is: LEADER
W20260810 07:24:27.413722 1131 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:27.413806 1131 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:27.413867 1130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:27.414816 1130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:27.414975 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:27.410242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:51509 (local address 127.0.0.1:33961)
0810 07:24:27.410376 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:27.410380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:27.410904 (+ 524us) server_negotiation.cc:407] Connection header received
0810 07:24:27.411261 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:27.411264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:27.411309 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:27.411364 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:27.411796 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:27.412310 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:27.413215 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:27.413443 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:27.413563 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:27.414447 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:27.414451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:27.414454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:27.414466 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:27.414505 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:27.414516 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:27.414614 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:27.414740 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 07:24:27.414773 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:24:27.420706 1130 catalog_manager.cc:1347] Generated new cluster ID: 5e96dbc50b774d4999bbac23e31e563d
I20260810 07:24:27.420781 1130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:27.445094 1130 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:27.446122 1130 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:27.450625 1130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d259b15264543f9bc5b74f7476273cb: Generated new TSK 0
I20260810 07:24:27.451426 1130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:27.460820 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 983
W20260810 07:24:27.477669 963 connection.cc:535] client connection to 127.0.0.1:33961 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33961): Connection reset by peer (error 104)
2026-08-10T07:24:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1658 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:27 dist-test-slave-44d2 krb5kdc[1140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:27 dist-test-slave-44d2 krb5kdc[1140](info): set up 2 sockets
Aug 10 07:24:27 dist-test-slave-44d2 krb5kdc[1140](info): commencing operation
krb5kdc: starting...
W20260810 07:24:28.778623 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:28 dist-test-slave-44d2 krb5kdc[1140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:28.907547 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40919
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:29.021399 1160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:29.021699 1160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:29.021775 1160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:29.025825 1160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:29.025915 1160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:29.025954 1160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:29.026044 1160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:29.026086 1160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:29.031352 1160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:29.032457 1160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:29.033594 1160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:24:29.042996 1160 server_base.cc:1034] running on GCE node
W20260810 07:24:29.042887 1169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:29.042887 1171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:29.042891 1174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:29.043744 1160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:29.044996 1160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:29.046168 1160 hybrid_clock.cc:648] HybridClock initialized: now 1786346669046137 us; error 42 us; skew 500 ppm
Aug 10 07:24:29 dist-test-slave-44d2 krb5kdc[1140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346669, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:29.052762 1160 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:29.053627 1160 webserver.cc:460] Webserver started at http://127.0.0.1:36423/ using document root <none> and password file <none>
I20260810 07:24:29.054162 1160 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:29.054220 1160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:29.054389 1160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:29.056427 1160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "72243cae5bc24ed48ed3787f5d1928f5"
format_stamp: "Formatted at 2026-08-10 07:24:29 on dist-test-slave-44d2"
server_key: "1e19ed89f5e75732871acb67562ac59a"
server_key_iv: "079a0efc67583d7c7cd07d65e9c1ab7b"
server_key_version: "encryptionkey@0"
I20260810 07:24:29.057045 1160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "72243cae5bc24ed48ed3787f5d1928f5"
format_stamp: "Formatted at 2026-08-10 07:24:29 on dist-test-slave-44d2"
server_key: "1e19ed89f5e75732871acb67562ac59a"
server_key_iv: "079a0efc67583d7c7cd07d65e9c1ab7b"
server_key_version: "encryptionkey@0"
I20260810 07:24:29.060861 1160 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:24:29.063264 1181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:29.064597 1160 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:24:29.064742 1160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "72243cae5bc24ed48ed3787f5d1928f5"
format_stamp: "Formatted at 2026-08-10 07:24:29 on dist-test-slave-44d2"
server_key: "1e19ed89f5e75732871acb67562ac59a"
server_key_iv: "079a0efc67583d7c7cd07d65e9c1ab7b"
server_key_version: "encryptionkey@0"
I20260810 07:24:29.064860 1160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:29.069482 1160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:29.072361 1160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:29.072614 1160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:29.079910 1160 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40919
I20260810 07:24:29.079941 1261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40919 every 8 connection(s)
I20260810 07:24:29.080924 1160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:29.083638 1263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:29.084157 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 1160
I20260810 07:24:29.084301 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:29.084674 1527 external_mini_cluster.cc:1428] Setting key 3433c7a3dfcd7d18ad30e14d7c00efb0
I20260810 07:24:29.089895 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:29.086113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:49977 (local address 127.0.0.1:40919)
0810 07:24:29.086559 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:29.086574 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:29.086612 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:24:29.087278 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:29.087295 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:29.087572 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:29.087919 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:29.088739 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:29.088743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:29.088756 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:29.088767 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:29.088819 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:29.088838 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:29.088993 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:29.089063 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 07:24:29.089243 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":132,"threads_started":1}
I20260810 07:24:29.090585 1263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5: Bootstrap starting.
I20260810 07:24:29.092921 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:29.093602 1263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:29.094502 1263 log.cc:824] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:29.096836 1263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5: No bootstrap required, opened a new log
I20260810 07:24:29.096978 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:29.095145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:46045 (local address 127.0.0.1:40919)
0810 07:24:29.095282 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:29.095288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:29.095477 (+ 189us) server_negotiation.cc:407] Connection header received
0810 07:24:29.095560 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:29.095562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:29.095595 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:29.095679 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:29.096567 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:29.096571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:29.096574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:29.096586 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:29.096614 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:29.096627 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:29.096722 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:29.096822 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 07:24:29.096865 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:24:29.100338 1263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72243cae5bc24ed48ed3787f5d1928f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } }
I20260810 07:24:29.100651 1263 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:29.100728 1263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72243cae5bc24ed48ed3787f5d1928f5, State: Initialized, Role: FOLLOWER
I20260810 07:24:29.101154 1263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72243cae5bc24ed48ed3787f5d1928f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } }
I20260810 07:24:29.101280 1263 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:29.101361 1263 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:29.101481 1263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:29.102418 1263 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72243cae5bc24ed48ed3787f5d1928f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } }
I20260810 07:24:29.102771 1263 leader_election.cc:302] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72243cae5bc24ed48ed3787f5d1928f5; no voters:
I20260810 07:24:29.103091 1263 leader_election.cc:288] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:29.103212 1281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:29.103505 1281 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [term 1 LEADER]: Becoming Leader. State: Replica: 72243cae5bc24ed48ed3787f5d1928f5, State: Running, Role: LEADER
I20260810 07:24:29.103868 1281 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72243cae5bc24ed48ed3787f5d1928f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } }
I20260810 07:24:29.104363 1263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:29.105835 1282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72243cae5bc24ed48ed3787f5d1928f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72243cae5bc24ed48ed3787f5d1928f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } } }
I20260810 07:24:29.105868 1283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72243cae5bc24ed48ed3787f5d1928f5. Latest consensus state: current_term: 1 leader_uuid: "72243cae5bc24ed48ed3787f5d1928f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72243cae5bc24ed48ed3787f5d1928f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } } }
I20260810 07:24:29.105960 1282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:29.105962 1283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:29.106796 1295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:29.109606 1295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:29.115173 1295 catalog_manager.cc:1347] Generated new cluster ID: a3b05140377b463ba2ae0c42d493f7ec
I20260810 07:24:29.115248 1295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:29.123252 1295 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:29.123567 1295 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:29.128232 1295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72243cae5bc24ed48ed3787f5d1928f5: Generated new TSK 0
I20260810 07:24:29.128912 1295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:29.135972 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 1160
2026-08-10T07:24:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1676 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:29 dist-test-slave-44d2 krb5kdc[1320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:29 dist-test-slave-44d2 krb5kdc[1320](info): set up 2 sockets
Aug 10 07:24:29 dist-test-slave-44d2 krb5kdc[1320](info): commencing operation
krb5kdc: starting...
W20260810 07:24:30.497605 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:30 dist-test-slave-44d2 krb5kdc[1320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:30.632285 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45733
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:30.748842 1341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:30.749104 1341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:30.749146 1341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:30.753019 1341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:30.753072 1341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:30.753090 1341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:30.753108 1341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:30.753122 1341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:30.758230 1341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45733
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:30.759217 1341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:30.760212 1341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:30.768850 1348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:30.769073 1341 server_base.cc:1034] running on GCE node
W20260810 07:24:30.768880 1349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:30.768853 1353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:30.769775 1341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:30.770859 1341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:30.772071 1341 hybrid_clock.cc:648] HybridClock initialized: now 1786346670772032 us; error 52 us; skew 500 ppm
Aug 10 07:24:30 dist-test-slave-44d2 krb5kdc[1320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346670, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:30.775007 1341 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:30.775926 1341 webserver.cc:460] Webserver started at http://127.0.0.1:40365/ using document root <none> and password file <none>
I20260810 07:24:30.776448 1341 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:30.776571 1341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:30.776759 1341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:30.778770 1341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "7af0c518a6324d93b5dea93303d11ceb"
format_stamp: "Formatted at 2026-08-10 07:24:30 on dist-test-slave-44d2"
server_key: "20865436b97e56181b6bd0b092f129fc"
server_key_iv: "402564c43c8079cd7d6b5cabc9901542"
server_key_version: "encryptionkey@0"
I20260810 07:24:30.779304 1341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "7af0c518a6324d93b5dea93303d11ceb"
format_stamp: "Formatted at 2026-08-10 07:24:30 on dist-test-slave-44d2"
server_key: "20865436b97e56181b6bd0b092f129fc"
server_key_iv: "402564c43c8079cd7d6b5cabc9901542"
server_key_version: "encryptionkey@0"
I20260810 07:24:30.783661 1341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:24:30.786511 1367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:30.787882 1341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:24:30.788014 1341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "7af0c518a6324d93b5dea93303d11ceb"
format_stamp: "Formatted at 2026-08-10 07:24:30 on dist-test-slave-44d2"
server_key: "20865436b97e56181b6bd0b092f129fc"
server_key_iv: "402564c43c8079cd7d6b5cabc9901542"
server_key_version: "encryptionkey@0"
I20260810 07:24:30.788110 1341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:30.826879 1341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:30.830529 1341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:30.830760 1341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:30.838320 1341 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45733
I20260810 07:24:30.838349 1443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45733 every 8 connection(s)
I20260810 07:24:30.839308 1341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:30.842267 1445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:30.848136 1445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb: Bootstrap starting.
I20260810 07:24:30.849311 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 1341
I20260810 07:24:30.849498 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:30.849833 1527 external_mini_cluster.cc:1428] Setting key 0aac7e1c93547c323141fa9ab8db03d6
I20260810 07:24:30.851282 1445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:30.852052 1445 log.cc:824] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:30.854386 1445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb: No bootstrap required, opened a new log
I20260810 07:24:30.857887 1445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af0c518a6324d93b5dea93303d11ceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260810 07:24:30.858194 1445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:30.858278 1445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af0c518a6324d93b5dea93303d11ceb, State: Initialized, Role: FOLLOWER
I20260810 07:24:30.858764 1445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af0c518a6324d93b5dea93303d11ceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260810 07:24:30.858924 1445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:30.859015 1445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:30.859127 1445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [term 0 FOLLOWER]: Advancing to term 1
Aug 10 07:24:30 dist-test-slave-44d2 krb5kdc[1320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:24:30.860010 1445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af0c518a6324d93b5dea93303d11ceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260810 07:24:30.860364 1445 leader_election.cc:302] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7af0c518a6324d93b5dea93303d11ceb; no voters:
I20260810 07:24:30.860756 1445 leader_election.cc:288] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:30.860841 1454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:30.861406 1454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [term 1 LEADER]: Becoming Leader. State: Replica: 7af0c518a6324d93b5dea93303d11ceb, State: Running, Role: LEADER
I20260810 07:24:30.861815 1454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af0c518a6324d93b5dea93303d11ceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260810 07:24:30.862171 1445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:30.863428 1456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7af0c518a6324d93b5dea93303d11ceb. Latest consensus state: current_term: 1 leader_uuid: "7af0c518a6324d93b5dea93303d11ceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af0c518a6324d93b5dea93303d11ceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } } }
I20260810 07:24:30.863567 1456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:30.863986 1464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:30.863950 1455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7af0c518a6324d93b5dea93303d11ceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af0c518a6324d93b5dea93303d11ceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } } }
I20260810 07:24:30.864068 1455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:30.866797 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:30.851024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:60479 (local address 127.0.0.1:45733)
0810 07:24:30.851547 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:30.851563 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:30.851598 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:24:30.852288 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:30.852307 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:30.852783 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:30.853181 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:30.854168 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:30.855430 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:30.856671 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:30.857621 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:30.857744 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:30.860738 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:30.860745 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:30.860762 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:30.860799 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:30.862930 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:30.863639 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:30.863645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:30.863651 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:30.863799 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:30.864324 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:30.864330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:30.864332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:30.864998 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:30.865231 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:30.865579 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 07:24:30.865836 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":161,"threads_started":1}
I20260810 07:24:30.867105 1464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:30.869897 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:30.874066 1464 catalog_manager.cc:1347] Generated new cluster ID: a76ee41f6cdf4928a846817d379fcd38
I20260810 07:24:30.874127 1464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:30.880860 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:30.872193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:48421 (local address 127.0.0.1:45733)
0810 07:24:30.872288 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:30.872294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:30.872361 (+ 67us) server_negotiation.cc:407] Connection header received
0810 07:24:30.872526 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:30.872531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:30.872573 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:30.872679 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:30.873322 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:30.873966 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:30.875097 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:30.875472 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:30.875605 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:30.877152 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:30.877157 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:30.877160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:30.877190 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:30.878889 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:30.879419 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:30.879423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:30.879425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:30.879514 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:30.879858 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:30.879861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:30.879863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:30.880077 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:30.880231 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:30.880537 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 07:24:30.880656 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:24:30.888391 1464 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:30.889331 1464 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:30.903154 1464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7af0c518a6324d93b5dea93303d11ceb: Generated new TSK 0
I20260810 07:24:30.903981 1464 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:30.908562 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 1341
W20260810 07:24:30.926369 1313 connection.cc:535] client connection to 127.0.0.1:45733 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45733): Connection reset by peer (error 104)
2026-08-10T07:24:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1771 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:30 dist-test-slave-44d2 krb5kdc[1490](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:30 dist-test-slave-44d2 krb5kdc[1490](info): set up 2 sockets
Aug 10 07:24:30 dist-test-slave-44d2 krb5kdc[1490](info): commencing operation
krb5kdc: starting...
W20260810 07:24:32.197605 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:32 dist-test-slave-44d2 krb5kdc[1490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:32.335076 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:32.451040 1511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:32.451330 1511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:32.451403 1511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:32.455329 1511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:32.455420 1511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:32.455461 1511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:32.455518 1511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:32.455555 1511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:32.460726 1511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38675
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:32.461789 1511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:32.462848 1511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:32.472040 1526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:32.472218 1511 server_base.cc:1034] running on GCE node
W20260810 07:24:32.472110 1522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:32.472041 1520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:32.472929 1511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:32.474100 1511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:32.475298 1511 hybrid_clock.cc:648] HybridClock initialized: now 1786346672475264 us; error 44 us; skew 500 ppm
Aug 10 07:24:32 dist-test-slave-44d2 krb5kdc[1490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346672, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:32.478153 1511 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:32.479162 1511 webserver.cc:460] Webserver started at http://127.0.0.1:34077/ using document root <none> and password file <none>
I20260810 07:24:32.479758 1511 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:32.479859 1511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:32.480108 1511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:32.482285 1511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "e089ba5588854993800e5b36e8c7e390"
format_stamp: "Formatted at 2026-08-10 07:24:32 on dist-test-slave-44d2"
server_key: "0107b3fff08db14e924cbabfc14a8849"
server_key_iv: "76111055c6a9fda8515f71dd58b16ce9"
server_key_version: "encryptionkey@0"
I20260810 07:24:32.482890 1511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "e089ba5588854993800e5b36e8c7e390"
format_stamp: "Formatted at 2026-08-10 07:24:32 on dist-test-slave-44d2"
server_key: "0107b3fff08db14e924cbabfc14a8849"
server_key_iv: "76111055c6a9fda8515f71dd58b16ce9"
server_key_version: "encryptionkey@0"
I20260810 07:24:32.486711 1511 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260810 07:24:32.489270 1547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:32.490458 1511 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:24:32.490612 1511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "e089ba5588854993800e5b36e8c7e390"
format_stamp: "Formatted at 2026-08-10 07:24:32 on dist-test-slave-44d2"
server_key: "0107b3fff08db14e924cbabfc14a8849"
server_key_iv: "76111055c6a9fda8515f71dd58b16ce9"
server_key_version: "encryptionkey@0"
I20260810 07:24:32.490727 1511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:32.499881 1511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:32.502684 1511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:32.502894 1511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:32.510380 1511 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38675
I20260810 07:24:32.510418 1620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38675 every 8 connection(s)
I20260810 07:24:32.511314 1511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:32.512142 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 1511
I20260810 07:24:32.512298 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:32.512683 1527 external_mini_cluster.cc:1428] Setting key 2b2d99d5daa79b64b8669095eb60a263
I20260810 07:24:32.514577 1621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:32.521389 1621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390: Bootstrap starting.
I20260810 07:24:32.522814 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:32.514108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:57819 (local address 127.0.0.1:38675)
0810 07:24:32.514523 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:32.514542 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:32.514599 (+ 57us) server_negotiation.cc:407] Connection header received
0810 07:24:32.515418 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:32.515439 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:32.515860 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:32.516217 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:32.516637 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:32.518150 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:32.519048 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:32.519749 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:32.519921 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:32.520853 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:32.520859 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:32.520874 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:32.521343 (+ 469us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:32.521412 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:32.521447 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:32.521593 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:32.521783 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 07:24:32.521909 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":124,"threads_started":1}
I20260810 07:24:32.524087 1621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:32.525022 1621 log.cc:824] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:32.526008 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:32.527082 1621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390: No bootstrap required, opened a new log
I20260810 07:24:32.531531 1621 raft_consensus.cc:348] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e089ba5588854993800e5b36e8c7e390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38675 } }
I20260810 07:24:32.531832 1621 raft_consensus.cc:374] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:32.531909 1621 raft_consensus.cc:729] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e089ba5588854993800e5b36e8c7e390, State: Initialized, Role: FOLLOWER
I20260810 07:24:32.532347 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:32.528051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:46221 (local address 127.0.0.1:38675)
0810 07:24:32.528184 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:32.528188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:32.528402 (+ 214us) server_negotiation.cc:407] Connection header received
0810 07:24:32.528540 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:32.528542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:32.528581 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:32.528656 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:32.528968 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:32.529469 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:32.530653 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:32.530856 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:32.530936 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:32.531779 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:32.531782 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:32.531784 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:32.531795 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:32.531820 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:32.531855 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:32.531958 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:32.532164 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 07:24:32.532202 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:24:32.532373 1621 consensus_queue.cc:260] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e089ba5588854993800e5b36e8c7e390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38675 } }
I20260810 07:24:32.532588 1621 raft_consensus.cc:388] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:32.532686 1621 raft_consensus.cc:482] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:32.532786 1621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:32.533674 1621 raft_consensus.cc:504] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e089ba5588854993800e5b36e8c7e390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38675 } }
I20260810 07:24:32.534013 1621 leader_election.cc:302] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e089ba5588854993800e5b36e8c7e390; no voters:
I20260810 07:24:32.534363 1621 leader_election.cc:288] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:32.534474 1636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:32.534734 1636 raft_consensus.cc:686] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [term 1 LEADER]: Becoming Leader. State: Replica: e089ba5588854993800e5b36e8c7e390, State: Running, Role: LEADER
I20260810 07:24:32.535079 1636 consensus_queue.cc:237] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e089ba5588854993800e5b36e8c7e390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38675 } }
I20260810 07:24:32.535601 1621 sys_catalog.cc:564] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:32.536803 1638 sys_catalog.cc:455] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e089ba5588854993800e5b36e8c7e390. Latest consensus state: current_term: 1 leader_uuid: "e089ba5588854993800e5b36e8c7e390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e089ba5588854993800e5b36e8c7e390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38675 } } }
I20260810 07:24:32.536950 1638 sys_catalog.cc:458] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:32.537243 1637 sys_catalog.cc:455] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e089ba5588854993800e5b36e8c7e390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e089ba5588854993800e5b36e8c7e390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38675 } } }
I20260810 07:24:32.537319 1646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:32.537349 1637 sys_catalog.cc:458] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:32.540818 1646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:32.546648 1646 catalog_manager.cc:1347] Generated new cluster ID: 562660763ecc4e7d866a5c1f7d6e41d0
I20260810 07:24:32.546727 1646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:32.563002 1646 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:32.564522 1646 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:32.570050 1646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e089ba5588854993800e5b36e8c7e390: Generated new TSK 0
I20260810 07:24:32.570751 1646 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:32.582778 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 1511
W20260810 07:24:32.596951 1487 connection.cc:535] client connection to 127.0.0.1:38675 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38675): Connection reset by peer (error 104)
2026-08-10T07:24:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1670 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:32 dist-test-slave-44d2 krb5kdc[1662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:32 dist-test-slave-44d2 krb5kdc[1662](info): set up 2 sockets
Aug 10 07:24:32 dist-test-slave-44d2 krb5kdc[1662](info): commencing operation
krb5kdc: starting...
W20260810 07:24:33.834574 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:33 dist-test-slave-44d2 krb5kdc[1662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:24:33.968158 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45907
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:34.085647 1682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:34.085909 1682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:34.085951 1682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:34.090066 1682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:34.090124 1682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:34.090144 1682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:34.090219 1682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:34.090250 1682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:34.095487 1682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45907
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:34.096552 1682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:34.097571 1682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:34.106297 1689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:34.106274 1693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:34.106352 1682 server_base.cc:1034] running on GCE node
W20260810 07:24:34.106263 1691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:34.107102 1682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:34.108106 1682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:34.109277 1682 hybrid_clock.cc:648] HybridClock initialized: now 1786346674109247 us; error 42 us; skew 500 ppm
Aug 10 07:24:34 dist-test-slave-44d2 krb5kdc[1662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346674, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:34.117480 1682 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:24:34.118430 1682 webserver.cc:460] Webserver started at http://127.0.0.1:43175/ using document root <none> and password file <none>
I20260810 07:24:34.118999 1682 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:34.119061 1682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:34.119283 1682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:34.121392 1682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "a73c673668cb4c1390f32f12438f4779"
format_stamp: "Formatted at 2026-08-10 07:24:34 on dist-test-slave-44d2"
server_key: "7ee9d00223c48ae80614636a1942723a"
server_key_iv: "726ef3f04dd1e7a4fc2d0b9caeddfca3"
server_key_version: "encryptionkey@0"
I20260810 07:24:34.121975 1682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "a73c673668cb4c1390f32f12438f4779"
format_stamp: "Formatted at 2026-08-10 07:24:34 on dist-test-slave-44d2"
server_key: "7ee9d00223c48ae80614636a1942723a"
server_key_iv: "726ef3f04dd1e7a4fc2d0b9caeddfca3"
server_key_version: "encryptionkey@0"
I20260810 07:24:34.125684 1682 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:24:34.128068 1705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:34.129372 1682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:24:34.129535 1682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "a73c673668cb4c1390f32f12438f4779"
format_stamp: "Formatted at 2026-08-10 07:24:34 on dist-test-slave-44d2"
server_key: "7ee9d00223c48ae80614636a1942723a"
server_key_iv: "726ef3f04dd1e7a4fc2d0b9caeddfca3"
server_key_version: "encryptionkey@0"
I20260810 07:24:34.129648 1682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:34.134246 1682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:34.137555 1682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:34.137768 1682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:34.145064 1682 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45907
I20260810 07:24:34.145102 1782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45907 every 8 connection(s)
I20260810 07:24:34.145980 1682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:34.148751 1783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:34.154429 1783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779: Bootstrap starting.
I20260810 07:24:34.155102 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 1682
I20260810 07:24:34.155243 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:34.155556 1527 external_mini_cluster.cc:1428] Setting key 54c3fa2809eea0c22c3e494033685810
I20260810 07:24:34.157397 1783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:34.158314 1783 log.cc:824] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:34.160373 1783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779: No bootstrap required, opened a new log
I20260810 07:24:34.161132 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:34.156888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:58667 (local address 127.0.0.1:45907)
0810 07:24:34.157379 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:34.157398 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:34.157446 (+ 48us) server_negotiation.cc:407] Connection header received
0810 07:24:34.158307 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:34.158333 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:34.158703 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:34.159058 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:34.159867 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:34.159872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:34.159885 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:34.159897 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:34.159951 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:34.159971 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:34.160099 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:34.160201 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 07:24:34.160370 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":116,"threads_started":1}
I20260810 07:24:34.163847 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:34.164119 1783 raft_consensus.cc:348] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73c673668cb4c1390f32f12438f4779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45907 } }
I20260810 07:24:34.164319 1783 raft_consensus.cc:374] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:34.164390 1783 raft_consensus.cc:729] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a73c673668cb4c1390f32f12438f4779, State: Initialized, Role: FOLLOWER
I20260810 07:24:34.164827 1783 consensus_queue.cc:260] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73c673668cb4c1390f32f12438f4779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45907 } }
I20260810 07:24:34.164951 1783 raft_consensus.cc:388] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:34.165036 1783 raft_consensus.cc:482] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:34.165154 1783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:34.166158 1783 raft_consensus.cc:504] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73c673668cb4c1390f32f12438f4779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45907 } }
I20260810 07:24:34.166566 1783 leader_election.cc:302] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a73c673668cb4c1390f32f12438f4779; no voters:
I20260810 07:24:34.166817 1783 leader_election.cc:288] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:34.166921 1797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:34.167227 1797 raft_consensus.cc:686] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [term 1 LEADER]: Becoming Leader. State: Replica: a73c673668cb4c1390f32f12438f4779, State: Running, Role: LEADER
I20260810 07:24:34.167563 1797 consensus_queue.cc:237] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73c673668cb4c1390f32f12438f4779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45907 } }
I20260810 07:24:34.168085 1783 sys_catalog.cc:564] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:34.169375 1799 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a73c673668cb4c1390f32f12438f4779" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73c673668cb4c1390f32f12438f4779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45907 } } }
I20260810 07:24:34.169517 1799 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:34.169889 1812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:34.169875 1800 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a73c673668cb4c1390f32f12438f4779. Latest consensus state: current_term: 1 leader_uuid: "a73c673668cb4c1390f32f12438f4779" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73c673668cb4c1390f32f12438f4779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45907 } } }
I20260810 07:24:34.170068 1800 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:34.171929 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:34.166207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:32817 (local address 127.0.0.1:45907)
0810 07:24:34.166323 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:34.166326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:34.170412 (+ 4086us) server_negotiation.cc:407] Connection header received
0810 07:24:34.170595 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:34.170598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:34.170636 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:34.170693 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:34.171540 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:34.171543 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:34.171546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:34.171557 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:34.171584 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:34.171597 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:34.171687 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:34.171769 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 07:24:34.171809 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:24:34.173286 1812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:34.178725 1812 catalog_manager.cc:1347] Generated new cluster ID: 97bedad373ae47ef9dc8f6768dfe91f0
I20260810 07:24:34.178804 1812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:34.199177 1812 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:34.199553 1812 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:34.212813 1812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a73c673668cb4c1390f32f12438f4779: Generated new TSK 0
I20260810 07:24:34.213830 1812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:34.227830 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 1682
2026-08-10T07:24:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1650 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:34 dist-test-slave-44d2 krb5kdc[1827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:34 dist-test-slave-44d2 krb5kdc[1827](info): set up 2 sockets
Aug 10 07:24:34 dist-test-slave-44d2 krb5kdc[1827](info): commencing operation
krb5kdc: starting...
W20260810 07:24:35.488914 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:35 dist-test-slave-44d2 krb5kdc[1827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:24:35.621474 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:43897
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:43897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:35.734175 1841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:35.734498 1841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:35.734579 1841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:35.738619 1841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:35.738727 1841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:35.738770 1841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:35.738807 1841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:35.738843 1841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:35.744112 1841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:43897
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:43897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:35.745254 1841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:35.746366 1841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:35.755415 1850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:35.755424 1848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:35.755396 1856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:35.755748 1841 server_base.cc:1034] running on GCE node
I20260810 07:24:35.756553 1841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:35.757893 1841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:35.759100 1841 hybrid_clock.cc:648] HybridClock initialized: now 1786346675759077 us; error 32 us; skew 500 ppm
Aug 10 07:24:35 dist-test-slave-44d2 krb5kdc[1827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346675, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:35.762130 1841 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:24:35.762988 1841 webserver.cc:460] Webserver started at http://127.1.125.254:43467/ using document root <none> and password file <none>
I20260810 07:24:35.763558 1841 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:35.763624 1841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:35.763801 1841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:35.765908 1841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "6ddba786988d4281b4d0d0fad9d630bd"
format_stamp: "Formatted at 2026-08-10 07:24:35 on dist-test-slave-44d2"
server_key: "d2fb922a9d9f926dcaa79fa195992817"
server_key_iv: "2ba7ff33f6249b50ead63fab261047f4"
server_key_version: "encryptionkey@0"
I20260810 07:24:35.766494 1841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "6ddba786988d4281b4d0d0fad9d630bd"
format_stamp: "Formatted at 2026-08-10 07:24:35 on dist-test-slave-44d2"
server_key: "d2fb922a9d9f926dcaa79fa195992817"
server_key_iv: "2ba7ff33f6249b50ead63fab261047f4"
server_key_version: "encryptionkey@0"
I20260810 07:24:35.770428 1841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:24:35.772765 1864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:35.773998 1841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:24:35.774137 1841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "6ddba786988d4281b4d0d0fad9d630bd"
format_stamp: "Formatted at 2026-08-10 07:24:35 on dist-test-slave-44d2"
server_key: "d2fb922a9d9f926dcaa79fa195992817"
server_key_iv: "2ba7ff33f6249b50ead63fab261047f4"
server_key_version: "encryptionkey@0"
I20260810 07:24:35.774243 1841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:35.785199 1841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:35.788893 1841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:35.789111 1841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:35.796175 1841 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:43897
I20260810 07:24:35.796185 1944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:43897 every 8 connection(s)
I20260810 07:24:35.797214 1841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:35.798228 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 1841
I20260810 07:24:35.798385 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:35.798700 1527 external_mini_cluster.cc:1428] Setting key f8d1b800b7b5b847e08db58bbfb3023d
I20260810 07:24:35.800395 1945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:35.806969 1945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd: Bootstrap starting.
Aug 10 07:24:35 dist-test-slave-44d2 krb5kdc[1827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:24:35.809440 1945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:35.810189 1945 log.cc:824] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:35.812099 1945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd: No bootstrap required, opened a new log
I20260810 07:24:35.814072 1950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:35.800022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44967 (local address 127.1.125.254:43897)
0810 07:24:35.800451 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:35.800471 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:35.800560 (+ 89us) server_negotiation.cc:407] Connection header received
0810 07:24:35.801264 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:35.801282 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:35.801617 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:35.801893 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:35.802844 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:35.803843 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:35.805069 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:35.805528 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:35.805645 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:35.808538 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:35.808546 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:35.808569 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:35.808612 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:35.810453 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:35.810996 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:35.811002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:35.811006 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:35.811133 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:35.811496 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:35.811505 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:35.811507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:35.811905 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:35.812162 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:35.812692 (+ 530us) server_negotiation.cc:299] Negotiation successful
0810 07:24:35.812998 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":163,"threads_started":1}
I20260810 07:24:35.815447 1945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ddba786988d4281b4d0d0fad9d630bd" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43897 } }
I20260810 07:24:35.815713 1945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:35.815766 1945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ddba786988d4281b4d0d0fad9d630bd, State: Initialized, Role: FOLLOWER
I20260810 07:24:35.816251 1945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ddba786988d4281b4d0d0fad9d630bd" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43897 } }
I20260810 07:24:35.816417 1945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:35.816533 1945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:35.816658 1945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:35.817394 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:35.817797 1945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ddba786988d4281b4d0d0fad9d630bd" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43897 } }
I20260810 07:24:35.818178 1945 leader_election.cc:302] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ddba786988d4281b4d0d0fad9d630bd; no voters:
I20260810 07:24:35.818557 1945 leader_election.cc:288] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:35.818676 1954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:35.818943 1954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [term 1 LEADER]: Becoming Leader. State: Replica: 6ddba786988d4281b4d0d0fad9d630bd, State: Running, Role: LEADER
I20260810 07:24:35.819269 1954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ddba786988d4281b4d0d0fad9d630bd" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43897 } }
I20260810 07:24:35.819931 1945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:35.820513 1963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ddba786988d4281b4d0d0fad9d630bd. Latest consensus state: current_term: 1 leader_uuid: "6ddba786988d4281b4d0d0fad9d630bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ddba786988d4281b4d0d0fad9d630bd" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43897 } } }
I20260810 07:24:35.820645 1963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:35.821012 1959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ddba786988d4281b4d0d0fad9d630bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ddba786988d4281b4d0d0fad9d630bd" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43897 } } }
I20260810 07:24:35.821122 1959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:35.821697 1968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:35.825834 1968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:35.829427 1950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:35.819647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59103 (local address 127.1.125.254:43897)
0810 07:24:35.819760 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:35.819764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:35.820334 (+ 570us) server_negotiation.cc:407] Connection header received
0810 07:24:35.821409 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:35.821412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:35.821466 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:35.821545 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:35.822535 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:35.823298 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:35.824222 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:35.824466 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:35.824641 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:35.826109 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:35.826113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:35.826116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:35.826141 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:35.827595 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:35.828106 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:35.828109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:35.828110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:35.828182 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:35.828558 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:35.828563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:35.828565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:35.828740 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:35.828898 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:35.829173 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 07:24:35.829266 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:24:35.832155 1968 catalog_manager.cc:1347] Generated new cluster ID: f2b4104a254746a49022b127cc4e23c4
I20260810 07:24:35.832223 1968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:35.859817 1968 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:35.860932 1968 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:35.868465 1968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ddba786988d4281b4d0d0fad9d630bd: Generated new TSK 0
I20260810 07:24:35.869253 1968 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:35.882740 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 1841
W20260810 07:24:35.898988 1824 connection.cc:535] client connection to 127.1.125.254:43897 recv error: Network error: failed to read from TLS socket (remote: 127.1.125.254:43897): Connection reset by peer (error 104)
2026-08-10T07:24:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1651 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:35 dist-test-slave-44d2 krb5kdc[1990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:35 dist-test-slave-44d2 krb5kdc[1990](info): set up 2 sockets
Aug 10 07:24:35 dist-test-slave-44d2 krb5kdc[1990](info): commencing operation
krb5kdc: starting...
W20260810 07:24:37.140017 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:37 dist-test-slave-44d2 krb5kdc[1990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:24:37.273710 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:43425
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:33887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:43425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:37.386850 2007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:37.387167 2007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:37.387246 2007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:37.391220 2007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:37.391317 2007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:37.391356 2007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:37.391414 2007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:37.391471 2007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:37.396647 2007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:33887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:43425
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:43425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:37.397711 2007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:37.398739 2007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:37.407732 2012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:37.407724 2014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:37.407752 2016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:37.408061 2007 server_base.cc:1034] running on GCE node
I20260810 07:24:37.408712 2007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:37.409816 2007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:37.410982 2007 hybrid_clock.cc:648] HybridClock initialized: now 1786346677410959 us; error 34 us; skew 500 ppm
Aug 10 07:24:37 dist-test-slave-44d2 krb5kdc[1990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346677, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:37.413873 2007 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:24:37.414811 2007 webserver.cc:460] Webserver started at http://127.1.125.254:34055/ using document root <none> and password file <none>
I20260810 07:24:37.415406 2007 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:37.415505 2007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:37.415752 2007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:37.417876 2007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "a35fd18fc11b45c29b46255c80ebe259"
format_stamp: "Formatted at 2026-08-10 07:24:37 on dist-test-slave-44d2"
server_key: "51676859f9d49dc867d1934273656386"
server_key_iv: "aba3ed6bf5f9c966d92e205a187bae3b"
server_key_version: "encryptionkey@0"
I20260810 07:24:37.418471 2007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "a35fd18fc11b45c29b46255c80ebe259"
format_stamp: "Formatted at 2026-08-10 07:24:37 on dist-test-slave-44d2"
server_key: "51676859f9d49dc867d1934273656386"
server_key_iv: "aba3ed6bf5f9c966d92e205a187bae3b"
server_key_version: "encryptionkey@0"
I20260810 07:24:37.422372 2007 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:24:37.424837 2028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:37.426064 2007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:24:37.426225 2007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "a35fd18fc11b45c29b46255c80ebe259"
format_stamp: "Formatted at 2026-08-10 07:24:37 on dist-test-slave-44d2"
server_key: "51676859f9d49dc867d1934273656386"
server_key_iv: "aba3ed6bf5f9c966d92e205a187bae3b"
server_key_version: "encryptionkey@0"
I20260810 07:24:37.426340 2007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:37.452859 2007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:37.455839 2007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:37.456065 2007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:37.463995 2007 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:43425
I20260810 07:24:37.464046 2101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:43425 every 8 connection(s)
I20260810 07:24:37.465065 2007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:37.468036 2102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:37.471469 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 2007
I20260810 07:24:37.471692 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:37.471990 1527 external_mini_cluster.cc:1428] Setting key 7b4d4273d3feb7e24dfbb968594f49ac
I20260810 07:24:37.474298 2102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259: Bootstrap starting.
I20260810 07:24:37.477174 2102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:37.478058 2102 log.cc:824] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:37.480317 2102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259: No bootstrap required, opened a new log
I20260810 07:24:37.482980 2106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:37.473604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54593 (local address 127.1.125.254:43425)
0810 07:24:37.474064 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:37.474084 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:37.474128 (+ 44us) server_negotiation.cc:407] Connection header received
0810 07:24:37.474938 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:37.474963 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:37.475413 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:37.475812 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:37.476248 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:37.477802 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:37.478944 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:37.479618 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:37.479780 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:37.480745 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:37.480754 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:37.480774 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:37.481275 (+ 501us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:37.481341 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:37.481373 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:37.481524 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:37.481656 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 07:24:37.481789 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":113,"threads_started":1}
I20260810 07:24:37.484315 2102 raft_consensus.cc:348] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35fd18fc11b45c29b46255c80ebe259" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43425 } }
I20260810 07:24:37.484599 2102 raft_consensus.cc:374] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:37.484673 2102 raft_consensus.cc:729] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35fd18fc11b45c29b46255c80ebe259, State: Initialized, Role: FOLLOWER
I20260810 07:24:37.485163 2102 consensus_queue.cc:260] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35fd18fc11b45c29b46255c80ebe259" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43425 } }
I20260810 07:24:37.485349 2102 raft_consensus.cc:388] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:37.485419 2102 raft_consensus.cc:482] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:37.485584 2102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:37.486241 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:37.486663 2102 raft_consensus.cc:504] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35fd18fc11b45c29b46255c80ebe259" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43425 } }
I20260810 07:24:37.487001 2102 leader_election.cc:302] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a35fd18fc11b45c29b46255c80ebe259; no voters:
I20260810 07:24:37.487329 2102 leader_election.cc:288] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:37.487421 2108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:37.487614 2108 raft_consensus.cc:686] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [term 1 LEADER]: Becoming Leader. State: Replica: a35fd18fc11b45c29b46255c80ebe259, State: Running, Role: LEADER
I20260810 07:24:37.487954 2108 consensus_queue.cc:237] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35fd18fc11b45c29b46255c80ebe259" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43425 } }
I20260810 07:24:37.488722 2102 sys_catalog.cc:564] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:37.489300 2108 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a35fd18fc11b45c29b46255c80ebe259" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35fd18fc11b45c29b46255c80ebe259" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43425 } } }
I20260810 07:24:37.489435 2108 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:37.489660 2108 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a35fd18fc11b45c29b46255c80ebe259. Latest consensus state: current_term: 1 leader_uuid: "a35fd18fc11b45c29b46255c80ebe259" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35fd18fc11b45c29b46255c80ebe259" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43425 } } }
I20260810 07:24:37.489753 2108 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:37.491271 2127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:37.493597 2106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:37.488533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32959 (local address 127.1.125.254:43425)
0810 07:24:37.488639 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:37.488643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:37.488864 (+ 221us) server_negotiation.cc:407] Connection header received
0810 07:24:37.489042 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:37.489045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:37.489089 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:37.489189 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:37.489735 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:37.490496 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:37.491682 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:37.492001 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:37.492116 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:37.493087 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:37.493091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:37.493095 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:37.493108 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:37.493135 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:37.493147 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:37.493239 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:37.493427 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 07:24:37.493461 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:24:37.494553 2127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:37.500851 2127 catalog_manager.cc:1347] Generated new cluster ID: 1c7046fe72f34b39bbef9bb424cd428d
I20260810 07:24:37.500931 2127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:37.516151 2127 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:37.517076 2127 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:37.521180 2127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a35fd18fc11b45c29b46255c80ebe259: Generated new TSK 0
I20260810 07:24:37.521845 2127 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:37.530998 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 2007
W20260810 07:24:37.548111 1986 connection.cc:535] client connection to 127.1.125.254:43425 recv error: Network error: failed to read from TLS socket (remote: 127.1.125.254:43425): Connection reset by peer (error 104)
2026-08-10T07:24:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1649 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:37 dist-test-slave-44d2 krb5kdc[2157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:37 dist-test-slave-44d2 krb5kdc[2157](info): set up 2 sockets
Aug 10 07:24:37 dist-test-slave-44d2 krb5kdc[2157](info): commencing operation
krb5kdc: starting...
W20260810 07:24:38.839661 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:38 dist-test-slave-44d2 krb5kdc[2157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:24:38.969640 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:35511
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:46375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:35511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:39.084625 2178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:39.085042 2178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:39.085160 2178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:39.089232 2178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:39.089323 2178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:39.089361 2178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:39.089422 2178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:39.089464 2178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:39.094842 2178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:46375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:35511
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:35511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:39.096035 2178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:39.097185 2178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:39.106604 2185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:39.106618 2189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:39.106915 2178 server_base.cc:1034] running on GCE node
W20260810 07:24:39.106595 2186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:39.107646 2178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:39.108997 2178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:39.110206 2178 hybrid_clock.cc:648] HybridClock initialized: now 1786346679110176 us; error 44 us; skew 500 ppm
Aug 10 07:24:39 dist-test-slave-44d2 krb5kdc[2157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346679, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:39.113571 2178 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:24:39.114472 2178 webserver.cc:460] Webserver started at http://127.1.125.254:38805/ using document root <none> and password file <none>
I20260810 07:24:39.115060 2178 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:39.115126 2178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:39.115370 2178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:39.117570 2178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "89838b5e29864f51870bed9526471637"
format_stamp: "Formatted at 2026-08-10 07:24:39 on dist-test-slave-44d2"
server_key: "3408b93bcdfe4ef8226924bc7e146ba6"
server_key_iv: "79641dbb5cdd947cede3e115ca1070e5"
server_key_version: "encryptionkey@0"
I20260810 07:24:39.118167 2178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "89838b5e29864f51870bed9526471637"
format_stamp: "Formatted at 2026-08-10 07:24:39 on dist-test-slave-44d2"
server_key: "3408b93bcdfe4ef8226924bc7e146ba6"
server_key_iv: "79641dbb5cdd947cede3e115ca1070e5"
server_key_version: "encryptionkey@0"
I20260810 07:24:39.122062 2178 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:24:39.124531 2200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:39.125824 2178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:24:39.125980 2178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "89838b5e29864f51870bed9526471637"
format_stamp: "Formatted at 2026-08-10 07:24:39 on dist-test-slave-44d2"
server_key: "3408b93bcdfe4ef8226924bc7e146ba6"
server_key_iv: "79641dbb5cdd947cede3e115ca1070e5"
server_key_version: "encryptionkey@0"
I20260810 07:24:39.126096 2178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:39.130748 2178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:39.137472 2178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:39.137691 2178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:39.145147 2178 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:35511
I20260810 07:24:39.145176 2280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:35511 every 8 connection(s)
I20260810 07:24:39.146202 2178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:39.146418 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 2178
I20260810 07:24:39.146582 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:39.146929 1527 external_mini_cluster.cc:1428] Setting key 1e229311e7d464d208430e96543e418c
I20260810 07:24:39.149523 2281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:39.152659 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:39.148370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35099 (local address 127.1.125.254:35511)
0810 07:24:39.148871 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:39.148892 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:39.148936 (+ 44us) server_negotiation.cc:407] Connection header received
0810 07:24:39.149803 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:39.149826 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:39.150181 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:39.150566 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:39.151232 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:39.151237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:39.151251 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:39.151262 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:39.151318 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:39.151345 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:39.151482 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:39.151553 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 07:24:39.151711 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":201,"threads_started":1}
I20260810 07:24:39.155479 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:39.156435 2281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637: Bootstrap starting.
I20260810 07:24:39.159248 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:39.157623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43079 (local address 127.1.125.254:35511)
0810 07:24:39.157741 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:39.157745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:39.157911 (+ 166us) server_negotiation.cc:407] Connection header received
0810 07:24:39.158037 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:39.158039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:39.158068 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:39.158133 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:39.158832 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:39.158835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:39.158837 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:39.158847 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:39.158872 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:39.158882 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:39.158951 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:39.159067 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 07:24:39.159098 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:24:39.159309 2281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:39.160199 2281 log.cc:824] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:39.162122 2281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637: No bootstrap required, opened a new log
I20260810 07:24:39.166050 2281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89838b5e29864f51870bed9526471637" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 35511 } }
I20260810 07:24:39.166296 2281 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:39.166376 2281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89838b5e29864f51870bed9526471637, State: Initialized, Role: FOLLOWER
I20260810 07:24:39.166810 2281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89838b5e29864f51870bed9526471637" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 35511 } }
I20260810 07:24:39.166960 2281 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:39.167033 2281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:39.167181 2281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:39.168063 2281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89838b5e29864f51870bed9526471637" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 35511 } }
I20260810 07:24:39.168412 2281 leader_election.cc:302] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89838b5e29864f51870bed9526471637; no voters:
I20260810 07:24:39.168826 2281 leader_election.cc:288] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:39.168938 2293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:39.169174 2293 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [term 1 LEADER]: Becoming Leader. State: Replica: 89838b5e29864f51870bed9526471637, State: Running, Role: LEADER
I20260810 07:24:39.169548 2293 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89838b5e29864f51870bed9526471637" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 35511 } }
I20260810 07:24:39.170123 2281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:39.171618 2294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89838b5e29864f51870bed9526471637. Latest consensus state: current_term: 1 leader_uuid: "89838b5e29864f51870bed9526471637" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89838b5e29864f51870bed9526471637" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 35511 } } }
I20260810 07:24:39.171748 2294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:39.172190 2311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:39.172547 2298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89838b5e29864f51870bed9526471637" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89838b5e29864f51870bed9526471637" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 35511 } } }
I20260810 07:24:39.172675 2298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:39.174945 2311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:39.180852 2311 catalog_manager.cc:1347] Generated new cluster ID: fee50bd2ab2849c7818403b5963fa270
I20260810 07:24:39.180924 2311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:39.194654 2311 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:39.194984 2311 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:39.201023 2311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89838b5e29864f51870bed9526471637: Generated new TSK 0
I20260810 07:24:39.201831 2311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:39.207011 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 2178
2026-08-10T07:24:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1676 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:39 dist-test-slave-44d2 krb5kdc[2327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:39 dist-test-slave-44d2 krb5kdc[2327](info): set up 2 sockets
Aug 10 07:24:39 dist-test-slave-44d2 krb5kdc[2327](info): commencing operation
krb5kdc: starting...
W20260810 07:24:40.464831 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:40 dist-test-slave-44d2 krb5kdc[2327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:24:40.599191 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:45601
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:44481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:45601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:40.724324 2349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:40.724686 2349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:40.724790 2349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:40.728827 2349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:40.728916 2349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:40.728955 2349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:40.729014 2349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:40.729049 2349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:40.734325 2349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:44481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:45601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:45601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:40.735400 2349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:40.736452 2349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:40.744865 2359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:40.744999 2349 server_base.cc:1034] running on GCE node
W20260810 07:24:40.744879 2358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:40.744879 2361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:40.745779 2349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:40.746896 2349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:40.748068 2349 hybrid_clock.cc:648] HybridClock initialized: now 1786346680748049 us; error 30 us; skew 500 ppm
Aug 10 07:24:40 dist-test-slave-44d2 krb5kdc[2327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346680, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:40.751328 2349 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:24:40.752223 2349 webserver.cc:460] Webserver started at http://127.1.125.254:41165/ using document root <none> and password file <none>
I20260810 07:24:40.752846 2349 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:40.752912 2349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:40.753082 2349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:40.755097 2349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "c6a339e2354f46719b7b96de127c18e3"
format_stamp: "Formatted at 2026-08-10 07:24:40 on dist-test-slave-44d2"
server_key: "06a8594e4f146d043859f181f221287c"
server_key_iv: "813d9bdf4ded8cc2b11992c2c60b3fe1"
server_key_version: "encryptionkey@0"
I20260810 07:24:40.755636 2349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "c6a339e2354f46719b7b96de127c18e3"
format_stamp: "Formatted at 2026-08-10 07:24:40 on dist-test-slave-44d2"
server_key: "06a8594e4f146d043859f181f221287c"
server_key_iv: "813d9bdf4ded8cc2b11992c2c60b3fe1"
server_key_version: "encryptionkey@0"
I20260810 07:24:40.759443 2349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260810 07:24:40.761914 2374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:40.763172 2349 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:24:40.763310 2349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "c6a339e2354f46719b7b96de127c18e3"
format_stamp: "Formatted at 2026-08-10 07:24:40 on dist-test-slave-44d2"
server_key: "06a8594e4f146d043859f181f221287c"
server_key_iv: "813d9bdf4ded8cc2b11992c2c60b3fe1"
server_key_version: "encryptionkey@0"
I20260810 07:24:40.763417 2349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:40.777504 2349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:40.780247 2349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:40.780450 2349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:40.788064 2349 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:45601
I20260810 07:24:40.788079 2451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:45601 every 8 connection(s)
I20260810 07:24:40.789074 2349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:40.791850 2454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:40.796690 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 2349
I20260810 07:24:40.796851 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:40.797169 1527 external_mini_cluster.cc:1428] Setting key 2c827364653e472e1273dbabd80b0256
I20260810 07:24:40.797667 2454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3: Bootstrap starting.
I20260810 07:24:40.800055 2454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:40.801071 2454 log.cc:824] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:40.803185 2454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3: No bootstrap required, opened a new log
Aug 10 07:24:40 dist-test-slave-44d2 krb5kdc[2327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:24:40.808077 2454 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a339e2354f46719b7b96de127c18e3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45601 } }
I20260810 07:24:40.808360 2454 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:40.808425 2454 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6a339e2354f46719b7b96de127c18e3, State: Initialized, Role: FOLLOWER
I20260810 07:24:40.808879 2454 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a339e2354f46719b7b96de127c18e3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45601 } }
I20260810 07:24:40.809042 2454 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:40.809134 2454 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:40.809233 2454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:40.810370 2454 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a339e2354f46719b7b96de127c18e3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45601 } }
I20260810 07:24:40.810739 2454 leader_election.cc:302] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6a339e2354f46719b7b96de127c18e3; no voters:
I20260810 07:24:40.811147 2454 leader_election.cc:288] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:40.811372 2465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:40.811657 2465 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [term 1 LEADER]: Becoming Leader. State: Replica: c6a339e2354f46719b7b96de127c18e3, State: Running, Role: LEADER
I20260810 07:24:40.811973 2465 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a339e2354f46719b7b96de127c18e3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45601 } }
I20260810 07:24:40.812448 2454 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:40.812752 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:40.798619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52979 (local address 127.1.125.254:45601)
0810 07:24:40.799040 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:40.799061 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:40.799106 (+ 45us) server_negotiation.cc:407] Connection header received
0810 07:24:40.799719 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:40.799734 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:40.800078 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:40.800413 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:40.801370 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:40.802382 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:40.803500 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:40.803955 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:40.804073 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:40.807110 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:40.807115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:40.807128 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:40.807155 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:40.809218 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:40.809749 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:40.809754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:40.809759 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:40.809873 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:40.810171 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:40.810173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:40.810175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:40.810619 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:40.810768 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:40.811465 (+ 697us) server_negotiation.cc:299] Negotiation successful
0810 07:24:40.811776 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":141,"threads_started":1}
I20260810 07:24:40.813629 2467 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6a339e2354f46719b7b96de127c18e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a339e2354f46719b7b96de127c18e3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45601 } } }
I20260810 07:24:40.813752 2467 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:40.814188 2469 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6a339e2354f46719b7b96de127c18e3. Latest consensus state: current_term: 1 leader_uuid: "c6a339e2354f46719b7b96de127c18e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a339e2354f46719b7b96de127c18e3" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 45601 } } }
I20260810 07:24:40.814284 2469 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:40.815855 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 07:24:40.818302 2489 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:40.818387 2489 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:40.818460 2481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:40.819125 2481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:40.824918 2481 catalog_manager.cc:1347] Generated new cluster ID: 1d2ca50c5eaa4802b1cc3b4310a418cd
I20260810 07:24:40.824995 2481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:40.826388 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:40.817989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57655 (local address 127.1.125.254:45601)
0810 07:24:40.818111 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:40.818117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:40.818318 (+ 201us) server_negotiation.cc:407] Connection header received
0810 07:24:40.818458 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:40.818461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:40.818506 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:40.818580 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:40.819250 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:40.819765 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:40.820647 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:40.820972 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:40.821084 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:40.822783 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:40.822787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:40.822789 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:40.822814 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:40.824417 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:40.825046 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:40.825049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:40.825051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:40.825133 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:40.825469 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:40.825472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:40.825474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:40.825710 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:40.825841 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:40.826126 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 07:24:40.826241 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:24:40.831822 2481 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:40.833066 2481 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:40.840108 2481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6a339e2354f46719b7b96de127c18e3: Generated new TSK 0
I20260810 07:24:40.840886 2481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:40.846061 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 2349
2026-08-10T07:24:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1641 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:40 dist-test-slave-44d2 krb5kdc[2504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:40 dist-test-slave-44d2 krb5kdc[2504](info): set up 2 sockets
Aug 10 07:24:40 dist-test-slave-44d2 krb5kdc[2504](info): commencing operation
krb5kdc: starting...
W20260810 07:24:42.162367 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:42 dist-test-slave-44d2 krb5kdc[2504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:24:42.305763 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38599
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:38599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:42.419103 2520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:42.419412 2520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:42.419489 2520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:42.423451 2520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:42.423555 2520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:42.423589 2520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:42.423635 2520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:42.423663 2520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:42.428875 2520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:38599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:38599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:42.429935 2520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:42.431002 2520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:42.439750 2526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:42.439812 2527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:42.439760 2530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:42.439870 2520 server_base.cc:1034] running on GCE node
I20260810 07:24:42.440671 2520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:42.441798 2520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:42.443001 2520 hybrid_clock.cc:648] HybridClock initialized: now 1786346682442981 us; error 31 us; skew 500 ppm
Aug 10 07:24:42 dist-test-slave-44d2 krb5kdc[2504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346682, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:42.446210 2520 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:24:42.447069 2520 webserver.cc:460] Webserver started at http://127.1.125.254:45047/ using document root <none> and password file <none>
I20260810 07:24:42.447587 2520 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:42.447651 2520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:42.447816 2520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:42.449903 2520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "043011c4dd1c4a8dabf05976ca6697aa"
format_stamp: "Formatted at 2026-08-10 07:24:42 on dist-test-slave-44d2"
server_key: "5b8f1d8b468f46a7917e7efe86bcdce4"
server_key_iv: "e899530199d9753b14dafcb7e3b99718"
server_key_version: "encryptionkey@0"
I20260810 07:24:42.450435 2520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "043011c4dd1c4a8dabf05976ca6697aa"
format_stamp: "Formatted at 2026-08-10 07:24:42 on dist-test-slave-44d2"
server_key: "5b8f1d8b468f46a7917e7efe86bcdce4"
server_key_iv: "e899530199d9753b14dafcb7e3b99718"
server_key_version: "encryptionkey@0"
I20260810 07:24:42.454129 2520 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:24:42.456467 2543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:42.457767 2520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:24:42.457873 2520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "043011c4dd1c4a8dabf05976ca6697aa"
format_stamp: "Formatted at 2026-08-10 07:24:42 on dist-test-slave-44d2"
server_key: "5b8f1d8b468f46a7917e7efe86bcdce4"
server_key_iv: "e899530199d9753b14dafcb7e3b99718"
server_key_version: "encryptionkey@0"
I20260810 07:24:42.457954 2520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:42.475276 2520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:42.478257 2520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:42.478416 2520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:42.485816 2520 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:38599
I20260810 07:24:42.486004 2619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:38599 every 8 connection(s)
I20260810 07:24:42.486863 2520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:42.490135 2620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:42.492841 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 2520
I20260810 07:24:42.492954 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:42.493307 1527 external_mini_cluster.cc:1428] Setting key 71a537a16ca56c8dbb5454d4ac96f6ce
I20260810 07:24:42.496887 2620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa: Bootstrap starting.
I20260810 07:24:42.499715 2620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:42.500878 2620 log.cc:824] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:42.502697 2624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:42.494717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53303 (local address 127.1.125.254:38599)
0810 07:24:42.495145 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:42.495161 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:42.495197 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:24:42.495845 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:42.495862 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:42.496202 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:42.496474 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:42.496895 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:42.497891 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:42.499250 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:42.499716 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:42.499882 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:42.500946 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:42.500951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:42.500963 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:42.501425 (+ 462us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:42.501484 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:42.501516 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:42.501633 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:42.501792 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 07:24:42.501997 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":133,"threads_started":1}
I20260810 07:24:42.503146 2620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa: No bootstrap required, opened a new log
I20260810 07:24:42.505788 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:42.507910 2620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043011c4dd1c4a8dabf05976ca6697aa" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38599 } }
I20260810 07:24:42.508215 2620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:42.508327 2620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 043011c4dd1c4a8dabf05976ca6697aa, State: Initialized, Role: FOLLOWER
I20260810 07:24:42.508977 2620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043011c4dd1c4a8dabf05976ca6697aa" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38599 } }
I20260810 07:24:42.509166 2620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:42.509238 2620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:42.509343 2620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:42.510422 2620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043011c4dd1c4a8dabf05976ca6697aa" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38599 } }
I20260810 07:24:42.510839 2620 leader_election.cc:302] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 043011c4dd1c4a8dabf05976ca6697aa; no voters:
I20260810 07:24:42.511230 2620 leader_election.cc:288] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:42.511340 2634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:42.511560 2634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [term 1 LEADER]: Becoming Leader. State: Replica: 043011c4dd1c4a8dabf05976ca6697aa, State: Running, Role: LEADER
I20260810 07:24:42.511970 2634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043011c4dd1c4a8dabf05976ca6697aa" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38599 } }
I20260810 07:24:42.512378 2624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:42.507831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48923 (local address 127.1.125.254:38599)
0810 07:24:42.507936 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:42.507939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:42.508175 (+ 236us) server_negotiation.cc:407] Connection header received
0810 07:24:42.508355 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:42.508359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:42.508438 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:42.508633 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:42.508873 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:42.509697 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:42.510511 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:42.510894 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:42.511035 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:42.511942 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:42.511946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:42.511949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:42.511965 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:42.511991 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:42.512002 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:42.512080 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:42.512172 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 07:24:42.512221 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:24:42.512729 2620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:42.513393 2634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 043011c4dd1c4a8dabf05976ca6697aa. Latest consensus state: current_term: 1 leader_uuid: "043011c4dd1c4a8dabf05976ca6697aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043011c4dd1c4a8dabf05976ca6697aa" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38599 } } }
I20260810 07:24:42.513535 2634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:42.513953 2644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:42.514362 2636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "043011c4dd1c4a8dabf05976ca6697aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043011c4dd1c4a8dabf05976ca6697aa" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 38599 } } }
I20260810 07:24:42.514477 2636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:42.517345 2644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:42.522939 2644 catalog_manager.cc:1347] Generated new cluster ID: 06c4ae68707e474597f123399df310e7
I20260810 07:24:42.523017 2644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:42.537091 2644 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:42.538005 2644 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:42.549953 2644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 043011c4dd1c4a8dabf05976ca6697aa: Generated new TSK 0
I20260810 07:24:42.550835 2644 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:42.558033 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 2520
2026-08-10T07:24:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1711 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:42 dist-test-slave-44d2 krb5kdc[2675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:42 dist-test-slave-44d2 krb5kdc[2675](info): set up 2 sockets
Aug 10 07:24:42 dist-test-slave-44d2 krb5kdc[2675](info): commencing operation
krb5kdc: starting...
W20260810 07:24:43.831352 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:43 dist-test-slave-44d2 krb5kdc[2675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:24:43.967698 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:42251
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:42251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:44.080734 2694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:44.081048 2694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:44.081139 2694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:44.085093 2694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:44.085186 2694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:44.085224 2694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:44.085263 2694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:44.085295 2694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:44.090457 2694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:42251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:42251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:44.091514 2694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:44.092657 2694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:44.101413 2706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:44.101432 2704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:44.101718 2694 server_base.cc:1034] running on GCE node
W20260810 07:24:44.101432 2703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:44.102331 2694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:44.103464 2694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:44.104629 2694 hybrid_clock.cc:648] HybridClock initialized: now 1786346684104615 us; error 24 us; skew 500 ppm
Aug 10 07:24:44 dist-test-slave-44d2 krb5kdc[2675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346684, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:44.107359 2694 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:24:44.108275 2694 webserver.cc:460] Webserver started at http://127.1.125.254:33031/ using document root <none> and password file <none>
I20260810 07:24:44.108901 2694 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:44.109016 2694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:44.109253 2694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:44.111285 2694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "ce374ea1d23c48e0873354f601d2fd41"
format_stamp: "Formatted at 2026-08-10 07:24:44 on dist-test-slave-44d2"
server_key: "0f6438f640abd87ffb18a976f7fdca50"
server_key_iv: "a7998f01b3eadda541002628bee330e2"
server_key_version: "encryptionkey@0"
I20260810 07:24:44.111860 2694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "ce374ea1d23c48e0873354f601d2fd41"
format_stamp: "Formatted at 2026-08-10 07:24:44 on dist-test-slave-44d2"
server_key: "0f6438f640abd87ffb18a976f7fdca50"
server_key_iv: "a7998f01b3eadda541002628bee330e2"
server_key_version: "encryptionkey@0"
I20260810 07:24:44.115605 2694 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:24:44.118486 2719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:44.119709 2694 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:24:44.119854 2694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "ce374ea1d23c48e0873354f601d2fd41"
format_stamp: "Formatted at 2026-08-10 07:24:44 on dist-test-slave-44d2"
server_key: "0f6438f640abd87ffb18a976f7fdca50"
server_key_iv: "a7998f01b3eadda541002628bee330e2"
server_key_version: "encryptionkey@0"
I20260810 07:24:44.119964 2694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:44.124580 2694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:44.127470 2694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:44.127669 2694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:44.135031 2694 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:42251
I20260810 07:24:44.135053 2795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:42251 every 8 connection(s)
I20260810 07:24:44.135955 2694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:44.138816 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:44.144193 2796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41: Bootstrap starting.
I20260810 07:24:44.144758 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 2694
I20260810 07:24:44.144917 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:44.145201 1527 external_mini_cluster.cc:1428] Setting key 254e12dc6a81f255d132835cddd7e07a
I20260810 07:24:44.146989 2796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:44.147768 2796 log.cc:824] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:44.149842 2796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41: No bootstrap required, opened a new log
I20260810 07:24:44.153213 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:44.146457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51739 (local address 127.1.125.254:42251)
0810 07:24:44.146884 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:44.146898 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:44.146934 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:24:44.147596 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:44.147612 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:44.147925 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:44.148245 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:44.151931 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:44.151936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:44.151947 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:44.151958 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:44.152018 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:44.152040 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:44.152162 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:44.152281 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 07:24:44.152438 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":127,"threads_started":1}
I20260810 07:24:44.153718 2796 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce374ea1d23c48e0873354f601d2fd41" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 42251 } }
I20260810 07:24:44.153934 2796 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:24:44.153996 2796 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce374ea1d23c48e0873354f601d2fd41, State: Initialized, Role: FOLLOWER
I20260810 07:24:44.154481 2796 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce374ea1d23c48e0873354f601d2fd41" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 42251 } }
I20260810 07:24:44.154644 2796 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:24:44.154735 2796 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:24:44.154857 2796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:44.155946 2796 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce374ea1d23c48e0873354f601d2fd41" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 42251 } }
I20260810 07:24:44.156144 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:44.156365 2796 leader_election.cc:302] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce374ea1d23c48e0873354f601d2fd41; no voters:
I20260810 07:24:44.156698 2796 leader_election.cc:288] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:24:44.156831 2808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:44.157063 2808 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [term 1 LEADER]: Becoming Leader. State: Replica: ce374ea1d23c48e0873354f601d2fd41, State: Running, Role: LEADER
I20260810 07:24:44.157434 2808 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce374ea1d23c48e0873354f601d2fd41" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 42251 } }
I20260810 07:24:44.157971 2796 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:44.158780 2808 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce374ea1d23c48e0873354f601d2fd41. Latest consensus state: current_term: 1 leader_uuid: "ce374ea1d23c48e0873354f601d2fd41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce374ea1d23c48e0873354f601d2fd41" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 42251 } } }
I20260810 07:24:44.158977 2808 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:44.159235 2809 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce374ea1d23c48e0873354f601d2fd41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce374ea1d23c48e0873354f601d2fd41" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 42251 } } }
I20260810 07:24:44.159328 2809 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:44.159439 2830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:44.162226 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:44.158520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42891 (local address 127.1.125.254:42251)
0810 07:24:44.158629 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:44.158633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:44.160266 (+ 1633us) server_negotiation.cc:407] Connection header received
0810 07:24:44.160431 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:44.160436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:44.160476 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:44.160628 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:44.161545 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:44.161550 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:44.161553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:24:44.161574 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:44.161614 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:24:44.161631 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:44.161719 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:44.162052 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 07:24:44.162096 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:24:44.163214 2830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:24:44.168928 2830 catalog_manager.cc:1347] Generated new cluster ID: abe37eaa180e475d9cd7414845a598a2
I20260810 07:24:44.168998 2830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:44.190668 2830 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:44.191026 2830 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:44.202668 2830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce374ea1d23c48e0873354f601d2fd41: Generated new TSK 0
I20260810 07:24:44.203563 2830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:24:44.217770 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 2694
2026-08-10T07:24:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1671 ms)
[----------] 18 tests from AuthTokenIssuingTest (30128 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:44 dist-test-slave-44d2 krb5kdc[2850](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:44 dist-test-slave-44d2 krb5kdc[2850](info): set up 2 sockets
Aug 10 07:24:44 dist-test-slave-44d2 krb5kdc[2850](info): commencing operation
krb5kdc: starting...
W20260810 07:24:45.478878 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:45 dist-test-slave-44d2 krb5kdc[2850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:24:45.611420 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:43295
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:36069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:43295,127.1.125.253:40729,127.1.125.252:41859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:45.725368 2868 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:24:45.725706 2868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:45.725780 2868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:45.725857 2868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:45.729866 2868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:45.729950 2868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:45.729995 2868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:45.730048 2868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:45.730086 2868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:45.735306 2868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:36069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:43295,127.1.125.253:40729,127.1.125.252:41859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:43295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:45.736425 2868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:45.737489 2868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:45.746363 2877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:45.746377 2874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:45.746523 2868 server_base.cc:1034] running on GCE node
W20260810 07:24:45.746450 2875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:45.747196 2868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:45.748327 2868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:45.749518 2868 hybrid_clock.cc:648] HybridClock initialized: now 1786346685749505 us; error 39 us; skew 500 ppm
Aug 10 07:24:45 dist-test-slave-44d2 krb5kdc[2850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346685, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:45.752274 2868 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:24:45.753202 2868 webserver.cc:460] Webserver started at http://127.1.125.254:35519/ using document root <none> and password file <none>
I20260810 07:24:45.753741 2868 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:45.753799 2868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:45.753973 2868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:45.756014 2868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "ff80353a802e4e8b8e1787e8cc1d73c7"
format_stamp: "Formatted at 2026-08-10 07:24:45 on dist-test-slave-44d2"
server_key: "70d4fd41f0a884a74b6001d0158a85bc"
server_key_iv: "37ca12dc25b5cc431c864168e8b2c01f"
server_key_version: "encryptionkey@0"
I20260810 07:24:45.756613 2868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "ff80353a802e4e8b8e1787e8cc1d73c7"
format_stamp: "Formatted at 2026-08-10 07:24:45 on dist-test-slave-44d2"
server_key: "70d4fd41f0a884a74b6001d0158a85bc"
server_key_iv: "37ca12dc25b5cc431c864168e8b2c01f"
server_key_version: "encryptionkey@0"
I20260810 07:24:45.760604 2868 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 07:24:45.762964 2893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:45.764261 2868 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:24:45.764415 2868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "ff80353a802e4e8b8e1787e8cc1d73c7"
format_stamp: "Formatted at 2026-08-10 07:24:45 on dist-test-slave-44d2"
server_key: "70d4fd41f0a884a74b6001d0158a85bc"
server_key_iv: "37ca12dc25b5cc431c864168e8b2c01f"
server_key_version: "encryptionkey@0"
I20260810 07:24:45.764578 2868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:45.777838 2868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:45.780709 2868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:45.780865 2868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:45.788414 2868 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:43295
I20260810 07:24:45.788770 2968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:43295 every 8 connection(s)
I20260810 07:24:45.789480 2868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:45.792712 2970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:45.796794 2970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:45.798532 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 2868
I20260810 07:24:45.798728 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:45.799126 1527 external_mini_cluster.cc:1428] Setting key 5afed76bda82ae8d614a2bfa3fa0af96
Aug 10 07:24:45 dist-test-slave-44d2 krb5kdc[2850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346685, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.253@KRBTEST.COM" created.
I20260810 07:24:45.813788 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:45.799887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51514 (local address 127.1.125.254:43295)
0810 07:24:45.800177 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:45.800187 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:45.800224 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:24:45.801600 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:45.801609 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:45.801873 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:45.802079 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:45.802935 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:45.803971 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:45.805175 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:45.805608 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:45.805796 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:45.808375 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:45.808388 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:45.808398 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:45.808423 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:45.810442 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:45.811002 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:45.811007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:45.811008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:45.811142 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:45.811477 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:45.811480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:45.811481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:45.811864 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:45.811978 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:45.812557 (+ 579us) server_negotiation.cc:299] Negotiation successful
0810 07:24:45.812844 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":98,"threads_started":1}
I20260810 07:24:45.813797 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:45.799075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:43295 (local address 127.0.0.1:51514)
0810 07:24:45.799714 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:45.799780 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:45.801211 (+ 1431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:45.802105 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:45.802110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:45.802487 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:45.802758 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:45.802771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:45.804110 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:45.804113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:45.805031 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:45.805042 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:45.805792 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:45.805797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:45.805896 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:45.806593 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:45.806611 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:45.808209 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:45.810568 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:45.810580 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:45.810592 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:45.810879 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:45.811245 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:45.811247 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:45.811249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:45.811376 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:45.812017 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:45.812028 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:45.812249 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 07:24:45.812439 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 07:24:45.812980 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":124,"thread_start_us":151,"threads_started":1}
I20260810 07:24:45.816291 2970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:45.817191 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:45.816817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:40729 (local address 127.0.0.1:38148)
0810 07:24:45.816991 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:45.817093 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 07:24:45.817452 2895 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.125.253:40729: connect: Connection refused (error 111)
I20260810 07:24:45.818370 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:45.818062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:40729 (local address 127.0.0.1:38154)
0810 07:24:45.818221 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:45.818290 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 07:24:45.818583 2970 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.253:40729: Network error: Client connection negotiation failed: client connection to 127.1.125.253:40729: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.253.keytab.
Entry for principal kudu/127.1.125.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.253.keytab.
I20260810 07:24:45.831002 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.253:40729
--webserver_interface=127.1.125.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.253.keytab
--principal=kudu/127.1.125.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:36069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:43295,127.1.125.253:40729,127.1.125.252:41859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:24:45.867888 2970 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } attempt: 1
I20260810 07:24:45.868948 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:45.868584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:40729 (local address 127.0.0.1:38170)
0810 07:24:45.868764 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:45.868843 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 07:24:45.869755 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:45.869532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:40729 (local address 127.0.0.1:38176)
0810 07:24:45.869632 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:45.869684 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 07:24:45.869910 2970 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.253:40729: Network error: Client connection negotiation failed: client connection to 127.1.125.253:40729: connect: Connection refused (error 111)
I20260810 07:24:45.938130 2970 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } attempt: 2
I20260810 07:24:45.939121 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:45.938714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:40729 (local address 127.0.0.1:38184)
0810 07:24:45.938930 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:45.939012 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 07:24:45.940044 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:45.939747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:40729 (local address 127.0.0.1:38196)
0810 07:24:45.939879 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:45.939942 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 07:24:45.940228 2970 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.253:40729: Network error: Client connection negotiation failed: client connection to 127.1.125.253:40729: connect: Connection refused (error 111)
W20260810 07:24:45.949360 2981 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:24:45.949674 2981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:45.949753 2981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:45.949854 2981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:45.954484 2981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:45.954545 2981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:45.954566 2981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:45.954586 2981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:45.954643 2981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:45.960624 2981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:36069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:43295,127.1.125.253:40729,127.1.125.252:41859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.253.keytab
--principal=kudu/127.1.125.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.253:40729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.125.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:45.961860 2981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:45.963001 2981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:45.972555 2989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:45.972581 2988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:45.972581 2991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:45.973014 2981 server_base.cc:1034] running on GCE node
I20260810 07:24:45.973688 2981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:45.974886 2981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:45.976089 2981 hybrid_clock.cc:648] HybridClock initialized: now 1786346685976076 us; error 32 us; skew 500 ppm
Aug 10 07:24:45 dist-test-slave-44d2 krb5kdc[2850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346685, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:45.978982 2981 init.cc:377] Logged in from keytab as kudu/127.1.125.253@KRBTEST.COM (short username kudu)
I20260810 07:24:45.979941 2981 webserver.cc:460] Webserver started at http://127.1.125.253:45499/ using document root <none> and password file <none>
I20260810 07:24:45.980624 2981 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:45.980728 2981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:45.980985 2981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:45.983278 2981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/data/instance:
uuid: "4932c716edfa42bc9702b00c1a46d953"
format_stamp: "Formatted at 2026-08-10 07:24:45 on dist-test-slave-44d2"
server_key: "9bfac7d6b482c71869742734e7efebb2"
server_key_iv: "24758bc842b2cad1e5b42681aac70499"
server_key_version: "encryptionkey@0"
I20260810 07:24:45.983891 2981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/wal/instance:
uuid: "4932c716edfa42bc9702b00c1a46d953"
format_stamp: "Formatted at 2026-08-10 07:24:45 on dist-test-slave-44d2"
server_key: "9bfac7d6b482c71869742734e7efebb2"
server_key_iv: "24758bc842b2cad1e5b42681aac70499"
server_key_version: "encryptionkey@0"
I20260810 07:24:45.987839 2981 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:24:45.990448 3002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:45.991729 2981 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:24:45.991883 2981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/wal
uuid: "4932c716edfa42bc9702b00c1a46d953"
format_stamp: "Formatted at 2026-08-10 07:24:45 on dist-test-slave-44d2"
server_key: "9bfac7d6b482c71869742734e7efebb2"
server_key_iv: "24758bc842b2cad1e5b42681aac70499"
server_key_version: "encryptionkey@0"
I20260810 07:24:45.992000 2981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:46.009863 2981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:46.013103 2981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:46.013325 2981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:46.020581 2981 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.253:40729
I20260810 07:24:46.020905 3082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.253:40729 every 8 connection(s)
I20260810 07:24:46.021603 2981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/data/info.pb
I20260810 07:24:46.024365 3083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:46.028016 3083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:46.028564 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 2981
I20260810 07:24:46.028697 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-1/wal/instance
I20260810 07:24:46.028971 1527 external_mini_cluster.cc:1428] Setting key b1d0edfc9ea8ed32435e0d1ecdc5c198
I20260810 07:24:46.031420 2970 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } attempt: 3
Aug 10 07:24:46 dist-test-slave-44d2 krb5kdc[2850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346685, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for kudu/127.1.125.253@KRBTEST.COM
Aug 10 07:24:46 dist-test-slave-44d2 krb5kdc[2850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346685, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.253@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.252@KRBTEST.COM" created.
I20260810 07:24:46.044171 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.031876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:40729 (local address 127.0.0.1:38212)
0810 07:24:46.032090 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.032107 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:46.032199 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:46.033121 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:46.033127 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:46.033336 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:46.033506 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.033517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.034776 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.034781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.035625 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.035637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.036646 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.036650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.036740 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.037484 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:46.037496 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:46.039190 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:46.041714 (+ 2524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.041717 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.041720 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.042070 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.042543 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.042546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.042548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.042640 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.043517 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:46.043521 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:46.043648 (+ 127us) client_negotiation.cc:769] Sending connection context
0810 07:24:46.043801 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 07:24:46.043981 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260810 07:24:46.045392 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.030063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51520 (local address 127.1.125.254:43295)
0810 07:24:46.030228 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.030233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.031196 (+ 963us) server_negotiation.cc:407] Connection header received
0810 07:24:46.032538 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.032544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.032624 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.032713 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.033935 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.034796 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.036131 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.036373 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.036494 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.040126 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.040129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.040131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.040152 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.042995 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.043734 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.043738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.043740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.043834 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.044194 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.044200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.044204 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.044443 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.044575 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.045119 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.045247 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 07:24:46.045550 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.030430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:43295 (local address 127.0.0.1:51520)
0810 07:24:46.031003 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.031075 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:46.032267 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:46.032796 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:46.032804 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:46.033273 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:46.033788 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.033801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.034956 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.034960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.035980 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.035990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.036554 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.036559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.036685 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.037444 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:46.037468 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:46.039970 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:46.043122 (+ 3152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.043131 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.043136 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.043560 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.043966 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.043971 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.043976 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.044080 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.044670 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:46.044679 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:46.044818 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 07:24:46.044979 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 07:24:46.045194 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":3,"spinlock_wait_cycles":7040,"thread_start_us":113,"threads_started":1}
I20260810 07:24:46.045553 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.031992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38212 (local address 127.1.125.253:40729)
0810 07:24:46.032462 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.032474 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.032536 (+ 62us) server_negotiation.cc:407] Connection header received
0810 07:24:46.032711 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.032716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.032928 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.033127 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.033628 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.034653 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.035772 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.036438 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.036681 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.039348 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.039355 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.039366 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.039389 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.041562 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.042239 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.042246 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.042253 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.042409 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.042761 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.042766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.042768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.043364 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.043500 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.043977 (+ 477us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.044317 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":360,"spinlock_wait_cycles":19072,"thread_start_us":139,"threads_started":1}
I20260810 07:24:46.047283 3083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:46.047907 2970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:46.048709 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.048378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42654)
0810 07:24:46.048593 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.048650 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260810 07:24:46.049619 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.049293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42658)
0810 07:24:46.049508 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.049557 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260810 07:24:46.049772 2970 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:41859: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Aug 10 07:24:46 dist-test-slave-44d2 krb5kdc[2850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346685, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.253@KRBTEST.COM for kudu/127.1.125.253@KRBTEST.COM
I20260810 07:24:46.057176 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.048098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:40729 (local address 127.0.0.1:38222)
0810 07:24:46.048344 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.048360 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:46.048440 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:46.049027 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:46.049029 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:46.049240 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:46.049366 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.049374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.050518 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.050521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.051126 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.051136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.051812 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.051815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.051896 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.052527 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:46.052546 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:46.053877 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:46.055733 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.055736 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.055737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.055930 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.056198 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.056200 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.056202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.056277 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.056734 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:46.056737 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:46.056816 (+ 79us) client_negotiation.cc:769] Sending connection context
0810 07:24:46.056904 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 07:24:46.057038 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260810 07:24:46.057345 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.048236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38222 (local address 127.1.125.253:40729)
0810 07:24:46.048640 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.048646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.048663 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:24:46.048702 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.048707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.048879 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.048985 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.049525 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.050383 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.051261 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.051690 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.051807 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.054009 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.054013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.054015 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.054040 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.055623 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.056030 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.056033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.056034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.056107 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.056374 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.056377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.056378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.056643 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.056724 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.057001 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.057132 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306}
I20260810 07:24:46.058004 3083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:46.058684 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.058424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42666)
0810 07:24:46.058533 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.058634 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 07:24:46.058920 3009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
I20260810 07:24:46.059896 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.059708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42676)
0810 07:24:46.059806 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.059849 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 07:24:46.060111 3083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:41859: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.252.keytab.
Entry for principal kudu/127.1.125.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.252.keytab.
I20260810 07:24:46.063463 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.252:41859
--webserver_interface=127.1.125.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.252.keytab
--principal=kudu/127.1.125.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:36069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:43295,127.1.125.253:40729,127.1.125.252:41859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:24:46.080932 2970 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } attempt: 1
I20260810 07:24:46.081825 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.081472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42678)
0810 07:24:46.081651 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.081725 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 07:24:46.082765 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.082497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42680)
0810 07:24:46.082638 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.082694 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 07:24:46.082906 2970 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:41859: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
I20260810 07:24:46.109404 3083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } attempt: 1
I20260810 07:24:46.110376 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.110030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42688)
0810 07:24:46.110214 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.110288 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 07:24:46.111174 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.110942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42692)
0810 07:24:46.111059 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.111120 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 07:24:46.111310 3083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:41859: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
I20260810 07:24:46.158129 2970 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } attempt: 2
I20260810 07:24:46.159194 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.158790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42704)
0810 07:24:46.159042 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.159111 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260810 07:24:46.160162 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.159869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42708)
0810 07:24:46.160041 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.160101 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260810 07:24:46.160312 2970 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:41859: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
I20260810 07:24:46.179502 3083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } attempt: 2
I20260810 07:24:46.180473 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.180112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42710)
0810 07:24:46.180321 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.180386 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 07:24:46.181588 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.181289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42726)
0810 07:24:46.181431 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.181490 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260810 07:24:46.181746 3083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:41859: Network error: Client connection negotiation failed: client connection to 127.1.125.252:41859: connect: Connection refused (error 111)
W20260810 07:24:46.182832 3092 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:24:46.183208 3092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:46.183277 3092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:46.183355 3092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:46.187613 3092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:46.187700 3092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:46.187747 3092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:46.187806 3092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:46.187844 3092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:46.193850 3092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:36069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:43295,127.1.125.253:40729,127.1.125.252:41859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.252.keytab
--principal=kudu/127.1.125.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.252:41859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.125.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:46.194988 3092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:46.196111 3092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:46.204968 3100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:46.204994 3102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:46.204983 3106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:46.205109 3092 server_base.cc:1034] running on GCE node
I20260810 07:24:46.206032 3092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:46.207199 3092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:46.208395 3092 hybrid_clock.cc:648] HybridClock initialized: now 1786346686208363 us; error 52 us; skew 500 ppm
Aug 10 07:24:46 dist-test-slave-44d2 krb5kdc[2850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346686, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:46.211184 3092 init.cc:377] Logged in from keytab as kudu/127.1.125.252@KRBTEST.COM (short username kudu)
I20260810 07:24:46.212137 3092 webserver.cc:460] Webserver started at http://127.1.125.252:45991/ using document root <none> and password file <none>
I20260810 07:24:46.212796 3092 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:46.212893 3092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:46.213135 3092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:46.215204 3092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/data/instance:
uuid: "725d16f713034ccfa081978a66a78dff"
format_stamp: "Formatted at 2026-08-10 07:24:46 on dist-test-slave-44d2"
server_key: "4eb0a83f5be50a9e5c0faf1ba99d3d1e"
server_key_iv: "4b1a47059d504a5ef64c4c134b6655a8"
server_key_version: "encryptionkey@0"
I20260810 07:24:46.215796 3092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/wal/instance:
uuid: "725d16f713034ccfa081978a66a78dff"
format_stamp: "Formatted at 2026-08-10 07:24:46 on dist-test-slave-44d2"
server_key: "4eb0a83f5be50a9e5c0faf1ba99d3d1e"
server_key_iv: "4b1a47059d504a5ef64c4c134b6655a8"
server_key_version: "encryptionkey@0"
I20260810 07:24:46.219616 3092 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:24:46.222144 3115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:46.223430 3092 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:24:46.223591 3092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/wal
uuid: "725d16f713034ccfa081978a66a78dff"
format_stamp: "Formatted at 2026-08-10 07:24:46 on dist-test-slave-44d2"
server_key: "4eb0a83f5be50a9e5c0faf1ba99d3d1e"
server_key_iv: "4b1a47059d504a5ef64c4c134b6655a8"
server_key_version: "encryptionkey@0"
I20260810 07:24:46.223706 3092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:46.239264 3092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:46.242275 3092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:46.242501 3092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:46.250072 3092 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.252:41859
I20260810 07:24:46.250391 3195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.252:41859 every 8 connection(s)
I20260810 07:24:46.251108 3092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/data/info.pb
I20260810 07:24:46.254033 3196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:46.258224 3196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:46.259498 2970 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } attempt: 3
I20260810 07:24:46.261296 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 3092
I20260810 07:24:46.261427 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346386457763-1527-0/minicluster-data/master-2/wal/instance
I20260810 07:24:46.261730 1527 external_mini_cluster.cc:1428] Setting key 649a821571cf20b47625853183b71734
Aug 10 07:24:46 dist-test-slave-44d2 krb5kdc[2850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346685, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for kudu/127.1.125.252@KRBTEST.COM
Aug 10 07:24:46 dist-test-slave-44d2 krb5kdc[2850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346686, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.252@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
Aug 10 07:24:46 dist-test-slave-44d2 krb5kdc[2850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.252@KRBTEST.COM
I20260810 07:24:46.273063 3083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } attempt: 3
I20260810 07:24:46.273720 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.260131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42740)
0810 07:24:46.260355 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.260372 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:46.260470 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:46.262822 (+ 2352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:46.262826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:46.263037 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:46.263142 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.263149 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.265128 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.265131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.265674 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.265682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.266452 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.266456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.266567 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.267233 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:46.267252 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:46.268720 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:46.271627 (+ 2907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.271631 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.271634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.271899 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.272275 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.272279 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.272281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.272377 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.273211 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:46.273214 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:46.273311 (+ 97us) client_negotiation.cc:769] Sending connection context
0810 07:24:46.273405 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 07:24:46.273549 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260810 07:24:46.274350 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.260282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51536 (local address 127.1.125.254:43295)
0810 07:24:46.260398 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.260403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.261452 (+ 1049us) server_negotiation.cc:407] Connection header received
0810 07:24:46.262749 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.262753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.262812 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.262870 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.263962 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.264575 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.266004 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.266321 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.266434 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.269888 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.269894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.269900 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.269925 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.271479 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.272119 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.272124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.272129 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.272217 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.272645 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.272648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.272650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.272876 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.272964 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.273551 (+ 587us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.273675 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:24:46.275698 3199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.260716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42740 (local address 127.1.125.252:41859)
0810 07:24:46.261297 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.261317 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.261361 (+ 44us) server_negotiation.cc:407] Connection header received
0810 07:24:46.262269 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.262297 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.262575 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.262890 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.263383 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.264996 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.265805 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.266321 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.266484 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.268871 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.268876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.268886 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.268908 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.271530 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.272025 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.272032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.272034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.272169 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.272477 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.272504 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.272506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.273065 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.274004 (+ 939us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.274158 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.274540 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":463,"spinlock_wait_cycles":30592,"thread_start_us":186,"threads_started":1}
I20260810 07:24:46.275847 3203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.262431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42750 (local address 127.1.125.252:41859)
0810 07:24:46.262749 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.262758 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.263205 (+ 447us) server_negotiation.cc:407] Connection header received
0810 07:24:46.263243 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.263246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.263286 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.263375 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.264384 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.265133 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.266254 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.266662 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.266791 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.270192 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.270195 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.270198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.270229 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.272520 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.272995 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.273002 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.273007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.273123 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.273449 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.273454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.273456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.273664 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.273860 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.274219 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.274601 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":132,"server-negotiator.queue_time_us":254,"spinlock_wait_cycles":45952,"thread_start_us":114,"threads_started":1}
I20260810 07:24:46.275698 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.260716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:43295 (local address 127.0.0.1:51536)
0810 07:24:46.261304 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.261358 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:46.262295 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:46.263087 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:46.263093 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:46.263586 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:46.263821 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.263837 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.264692 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.264696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.265873 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.265885 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.266699 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.266701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.266893 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.267483 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:46.267510 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:46.269717 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:46.271611 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.271621 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.271632 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.271978 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.272342 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.272346 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.272348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.272509 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.273057 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:46.273063 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:46.273288 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 07:24:46.274747 (+ 1459us) client_negotiation.cc:240] Negotiation successful
0810 07:24:46.274952 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":85,"thread_start_us":140,"threads_started":1}
I20260810 07:24:46.277244 3196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:46.280870 2970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7: Bootstrap starting.
Aug 10 07:24:46 dist-test-slave-44d2 krb5kdc[2850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346685, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.253@KRBTEST.COM for kudu/127.1.125.252@KRBTEST.COM
Aug 10 07:24:46 dist-test-slave-44d2 krb5kdc[2850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346686, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.252@KRBTEST.COM for kudu/127.1.125.253@KRBTEST.COM
I20260810 07:24:46.284610 2970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:46.285859 2970 log.cc:824] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:46.286726 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.273539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42756)
0810 07:24:46.273832 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.273848 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:46.273937 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:46.275334 (+ 1397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:46.275340 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:46.275526 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:46.275617 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.275623 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.278346 (+ 2723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.278353 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.278905 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.278914 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.279542 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.279546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.279660 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.280272 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:46.280286 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:46.281752 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:46.284812 (+ 3060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.284815 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.284817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.285097 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.285454 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.285457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.285459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.285551 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.286095 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:46.286098 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:46.286208 (+ 110us) client_negotiation.cc:769] Sending connection context
0810 07:24:46.286386 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 07:24:46.286534 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209}
I20260810 07:24:46.287647 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.273804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.1.125.252:41859)
0810 07:24:46.275120 (+ 1316us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.275127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.275138 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:24:46.275165 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.275167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.275199 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.275305 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.275737 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.277451 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.279065 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.279406 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.279506 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.282566 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.282569 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.282572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.282596 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.284641 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.285230 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.285233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.285235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.285343 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.285660 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.285662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.285664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.285871 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.287271 (+ 1400us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.287349 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.287462 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1261,"thread_start_us":97,"threads_started":1}
I20260810 07:24:46.288329 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.277827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38234 (local address 127.1.125.253:40729)
0810 07:24:46.277978 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.277984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.278216 (+ 232us) server_negotiation.cc:407] Connection header received
0810 07:24:46.278386 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.278389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.278436 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.278490 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.279065 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.279746 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.280707 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.281069 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.281156 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.283945 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.283949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.283952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.283979 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.285862 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.286639 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.286643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.286646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.286745 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.287148 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.287152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.287154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.287454 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.287560 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.288006 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.288132 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 07:24:46 dist-test-slave-44d2 krb5kdc[2850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:24:46.288570 2970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7: No bootstrap required, opened a new log
I20260810 07:24:46.288897 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.277759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:40729 (local address 127.0.0.1:38234)
0810 07:24:46.278143 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.278159 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:46.278281 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:46.278648 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:46.278651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:46.278827 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:46.278935 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.278941 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.279856 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.279861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.280564 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.280576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.281303 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.281307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.281412 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.281880 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:46.281899 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:46.283787 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:46.286033 (+ 2246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.286037 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.286039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.286323 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.286887 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.286891 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.286893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.287004 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.287651 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:46.287654 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:46.287768 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 07:24:46.288511 (+ 743us) client_negotiation.cc:240] Negotiation successful
0810 07:24:46.288684 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303}
I20260810 07:24:46.290009 3196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:46.291046 3083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953: Bootstrap starting.
I20260810 07:24:46.292846 2970 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } }
I20260810 07:24:46.293073 2970 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff80353a802e4e8b8e1787e8cc1d73c7, State: Initialized, Role: FOLLOWER
I20260810 07:24:46.293375 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.280690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51548 (local address 127.1.125.254:43295)
0810 07:24:46.280797 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.280801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.280812 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:24:46.280843 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.280845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.280886 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.280979 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.282360 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.283650 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.285416 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.285759 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.285832 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.289186 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.289190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.289192 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.289220 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.291248 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.291857 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.291863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.291868 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.291979 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.292297 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.292301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.292303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.292617 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.292816 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.293075 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.293203 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:24:46.293411 2970 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } }
I20260810 07:24:46.294458 3211 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } } }
I20260810 07:24:46.294610 3211 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:46.295390 2970 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:46.296836 3083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:46.297901 3083 log.cc:824] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953: Log is configured to *not* fsync() on all Append() calls
Aug 10 07:24:46 dist-test-slave-44d2 krb5kdc[2850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346686, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.252@KRBTEST.COM for kudu/127.1.125.252@KRBTEST.COM
I20260810 07:24:46.300282 3083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953: No bootstrap required, opened a new log
Aug 10 07:24:46 dist-test-slave-44d2 krb5kdc[2850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.253@KRBTEST.COM
I20260810 07:24:46.304512 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.290879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:41859 (local address 127.0.0.1:42760)
0810 07:24:46.291382 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:46.291411 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:46.291500 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:46.293746 (+ 2246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:46.293750 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:46.293953 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:46.294055 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.294065 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.295205 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.295209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.295845 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:46.295855 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.296447 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.296451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:46.296567 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.297086 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:46.297103 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:46.298746 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:46.302568 (+ 3822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.302571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.302574 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.302864 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.303243 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:46.303246 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:46.303249 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:46.303335 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:46.303857 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:46.303861 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:46.303981 (+ 120us) client_negotiation.cc:769] Sending connection context
0810 07:24:46.304073 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 07:24:46.304226 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":137,"threads_started":1}
I20260810 07:24:46.300480 2868 master_runner.cc:186] Error getting master registration for 127.1.125.253:40729: OK, instance_id { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" instance_seqno: 1786346686018505 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:24:46.304914 2868 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:24:46.305308 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.292637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.1.125.252:41859)
0810 07:24:46.293516 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.293522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.293536 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:24:46.293572 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.293575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.293625 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.293732 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.294188 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.295050 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.295967 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.296319 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.296793 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.298873 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.298877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.298880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.298916 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.302419 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.303010 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.303016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.303021 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.303117 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.303447 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.303454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.303459 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.303732 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.304852 (+ 1120us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.304955 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.305098 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":796}
I20260810 07:24:46.305205 3083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } }
I20260810 07:24:46.305691 3083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4932c716edfa42bc9702b00c1a46d953, State: Initialized, Role: FOLLOWER
I20260810 07:24:46.306108 3083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } }
I20260810 07:24:46.307566 3083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 07:24:46.307976 3226 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:46.308061 3226 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:46.308795 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.294675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.1.125.253:40729)
0810 07:24:46.294801 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.294805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.294986 (+ 181us) server_negotiation.cc:407] Connection header received
0810 07:24:46.295164 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.295170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.295238 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.295322 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.296130 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.297348 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.298668 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.299033 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.299502 (+ 469us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.304056 (+ 4554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.304063 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.304068 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.304096 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.306164 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.306777 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.306781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.306782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.306876 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.307216 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.307219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.307221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.307491 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.307689 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.308448 (+ 759us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.308612 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 07:24:46.309003 3196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff: Bootstrap starting.
I20260810 07:24:46.309338 3230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } } }
I20260810 07:24:46.309545 3230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:46.312317 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:46.312319 3196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:46.313406 3196 log.cc:824] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:46.313715 2935 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51548
I20260810 07:24:46.313930 2935 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 07:24:46.314051 2935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:24:46.315119 3251 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:46.315255 3251 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:46.315445 2935 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } }
I20260810 07:24:46.316386 2935 leader_election.cc:288] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [CANDIDATE]: Term 1 election: Requested vote from peers 4932c716edfa42bc9702b00c1a46d953 (127.1.125.253:40729), 725d16f713034ccfa081978a66a78dff (127.1.125.252:41859)
I20260810 07:24:46.316551 3196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff: No bootstrap required, opened a new log
I20260810 07:24:46.317561 3045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4932c716edfa42bc9702b00c1a46d953"
I20260810 07:24:46.317786 3045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:46.319157 3045 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff80353a802e4e8b8e1787e8cc1d73c7 in term 1.
I20260810 07:24:46.319641 2895 leader_election.cc:302] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932c716edfa42bc9702b00c1a46d953, ff80353a802e4e8b8e1787e8cc1d73c7; no voters:
I20260810 07:24:46.319989 3211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:46.320230 3211 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [term 1 LEADER]: Becoming Leader. State: Replica: ff80353a802e4e8b8e1787e8cc1d73c7, State: Running, Role: LEADER
I20260810 07:24:46.316772 2981 master_runner.cc:186] Error getting master registration for 127.1.125.252:41859: OK, instance_id { permanent_uuid: "725d16f713034ccfa081978a66a78dff" instance_seqno: 1786346686247928 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:24:46.320595 2981 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:24:46.320619 3211 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } }
I20260810 07:24:46.321839 3196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } }
I20260810 07:24:46.322101 3196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 725d16f713034ccfa081978a66a78dff, State: Initialized, Role: FOLLOWER
I20260810 07:24:46.322495 3196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } }
I20260810 07:24:46.324095 3196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:46.324412 3211 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff80353a802e4e8b8e1787e8cc1d73c7. Latest consensus state: current_term: 1 leader_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } } }
I20260810 07:24:46.324805 3211 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:46.325269 3153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "725d16f713034ccfa081978a66a78dff"
I20260810 07:24:46.325878 3153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:46.327458 3153 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff80353a802e4e8b8e1787e8cc1d73c7 in term 1.
I20260810 07:24:46.328246 3261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } } }
I20260810 07:24:46.328780 3268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:46.328727 3261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:46.329757 3268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260810 07:24:46.331786 3291 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:46.331877 3291 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:46.335187 3153 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff [term 1 FOLLOWER]: Refusing update from remote peer ff80353a802e4e8b8e1787e8cc1d73c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:46.335345 3045 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953 [term 1 FOLLOWER]: Refusing update from remote peer ff80353a802e4e8b8e1787e8cc1d73c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:46.336231 3256 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:46.336675 3256 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:46.345410 3261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff80353a802e4e8b8e1787e8cc1d73c7. Latest consensus state: current_term: 1 leader_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } } }
I20260810 07:24:46.345574 3261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:46.345832 3230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff80353a802e4e8b8e1787e8cc1d73c7. Latest consensus state: current_term: 1 leader_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } } }
I20260810 07:24:46.345978 3230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:46.354465 3211 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } } }
I20260810 07:24:46.354620 3211 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:46.356148 3261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } } }
I20260810 07:24:46.356271 3261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:46.357069 3296 mvcc.cc:205] Tried to move back new op lower bound from 7316876027222155264 to 7316876027180896256. Current Snapshot: MvccSnapshot[applied={T|T < 7316876027222155264}]
I20260810 07:24:46.359817 3256 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } } }
I20260810 07:24:46.359944 3256 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:46.360762 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.324753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42764 (local address 127.1.125.252:41859)
0810 07:24:46.325051 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.325056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.336674 (+ 11618us) server_negotiation.cc:407] Connection header received
0810 07:24:46.336896 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.336900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.336953 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.337015 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.338588 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.339470 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.343177 (+ 3707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.343574 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.343667 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.349699 (+ 6032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.349704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.349708 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.349741 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.352009 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.356318 (+ 4309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.356321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.356324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.356443 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.357308 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.357313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.357315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.357591 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.357720 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.360416 (+ 2696us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.360586 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198}
I20260810 07:24:46.361038 3300 mvcc.cc:205] Tried to move back new op lower bound from 7316876027222155264 to 7316876027180896256. Current Snapshot: MvccSnapshot[applied={T|T < 7316876027222155264}]
I20260810 07:24:46.361172 3230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff80353a802e4e8b8e1787e8cc1d73c7" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43295 } } peers { permanent_uuid: "4932c716edfa42bc9702b00c1a46d953" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 40729 } } peers { permanent_uuid: "725d16f713034ccfa081978a66a78dff" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 41859 } } }
I20260810 07:24:46.361308 3230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:46.361821 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.324869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38254 (local address 127.1.125.253:40729)
0810 07:24:46.325173 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.325182 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.336830 (+ 11648us) server_negotiation.cc:407] Connection header received
0810 07:24:46.337093 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.337099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.337160 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.337214 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.340997 (+ 3783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.341850 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.344556 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.344944 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.345038 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.352682 (+ 7644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.352686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.352690 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.352723 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.354650 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.356848 (+ 2198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.356853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.356855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.356957 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.359460 (+ 2503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.359465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.359467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.359713 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.359833 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.361518 (+ 1685us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.361664 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260810 07:24:46.363070 3268 catalog_manager.cc:1347] Generated new cluster ID: decbbffe0285411192bb852eef995c19
I20260810 07:24:46.363126 3268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:46.366474 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:46.324604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51550 (local address 127.1.125.254:43295)
0810 07:24:46.324731 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.324735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.334555 (+ 9820us) server_negotiation.cc:407] Connection header received
0810 07:24:46.334600 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.334604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.334657 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.334740 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.346571 (+ 11831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.347326 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.349046 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:46.349388 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:46.349474 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:46.362331 (+ 12857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:46.362336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:46.362342 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:46.362379 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:46.364451 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.365007 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.365011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.365013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.365105 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:46.365436 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:46.365440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:46.365442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:46.365666 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:46.365779 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:46.366163 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 07:24:46.366295 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:24:46.388590 3268 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:46.389904 3268 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:46.405773 3268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff80353a802e4e8b8e1787e8cc1d73c7: Generated new TSK 0
I20260810 07:24:46.406684 3268 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:24:46.442041 2977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:46.439220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51554 (local address 127.1.125.254:43295)
0810 07:24:46.439366 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.439372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.440788 (+ 1416us) server_negotiation.cc:407] Connection header received
0810 07:24:46.441025 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.441030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.441086 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.441145 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.441869 (+ 724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51554: BlockingRecv error: recv got EOF from 127.0.0.1:51554 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 07:24:46.443806 3205 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:46.440096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42774 (local address 127.1.125.252:41859)
0810 07:24:46.440225 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.440232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.442786 (+ 2554us) server_negotiation.cc:407] Connection header received
0810 07:24:46.442999 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.443003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.443061 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.443121 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.443669 (+ 548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42774: BlockingRecv error: recv got EOF from 127.0.0.1:42774 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 07:24:46.446147 3088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:46.444504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38268 (local address 127.1.125.253:40729)
0810 07:24:46.444647 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.444651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.445138 (+ 487us) server_negotiation.cc:407] Connection header received
0810 07:24:46.445331 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.445334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.445384 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.445439 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.446005 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38268: BlockingRecv error: recv got EOF from 127.0.0.1:38268 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 07:24:46.454738 3088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:46.452918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38282 (local address 127.1.125.253:40729)
0810 07:24:46.453034 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.453039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.453638 (+ 599us) server_negotiation.cc:407] Connection header received
0810 07:24:46.453895 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.453898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.454003 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.454062 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.454628 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38282: BlockingRecv error: recv got EOF from 127.0.0.1:38282 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260810 07:24:46.460258 3088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:46.458650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38292 (local address 127.1.125.253:40729)
0810 07:24:46.458757 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.458762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.459282 (+ 520us) server_negotiation.cc:407] Connection header received
0810 07:24:46.459500 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.459504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.459557 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.459613 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.460147 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38292: BlockingRecv error: recv got EOF from 127.0.0.1:38292 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 07:24:46.466607 3088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:46.464849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38294 (local address 127.1.125.253:40729)
0810 07:24:46.464968 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.464973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.465537 (+ 564us) server_negotiation.cc:407] Connection header received
0810 07:24:46.465754 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.465758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.465813 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.465870 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.466487 (+ 617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38294: BlockingRecv error: recv got EOF from 127.0.0.1:38294 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 07:24:46.474960 3088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:46.473239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38302 (local address 127.1.125.253:40729)
0810 07:24:46.473385 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.473390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.473937 (+ 547us) server_negotiation.cc:407] Connection header received
0810 07:24:46.474148 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.474152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.474206 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.474265 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.474841 (+ 576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38302: BlockingRecv error: recv got EOF from 127.0.0.1:38302 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 07:24:46.487464 3088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:46.485609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38308 (local address 127.1.125.253:40729)
0810 07:24:46.485740 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.485746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.486323 (+ 577us) server_negotiation.cc:407] Connection header received
0810 07:24:46.486567 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.486571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.486631 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.486690 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.487323 (+ 633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38308: BlockingRecv error: recv got EOF from 127.0.0.1:38308 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 07:24:46.508699 3088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:46.506639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38324 (local address 127.1.125.253:40729)
0810 07:24:46.506782 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.506788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.507487 (+ 699us) server_negotiation.cc:407] Connection header received
0810 07:24:46.507739 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.507744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.507808 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.507871 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.508554 (+ 683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38324: BlockingRecv error: recv got EOF from 127.0.0.1:38324 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 07:24:46.545873 3088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:46.543949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38326 (local address 127.1.125.253:40729)
0810 07:24:46.544093 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.544098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.544769 (+ 671us) server_negotiation.cc:407] Connection header received
0810 07:24:46.544993 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.544998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.545057 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.545115 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.545738 (+ 623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38326: BlockingRecv error: recv got EOF from 127.0.0.1:38326 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260810 07:24:46.614024 3088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:46.612812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38342 (local address 127.1.125.253:40729)
0810 07:24:46.612999 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.613004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.613109 (+ 105us) server_negotiation.cc:407] Connection header received
0810 07:24:46.613246 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.613249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.613298 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.613375 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.613886 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38342: BlockingRecv error: recv got EOF from 127.0.0.1:38342 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 07:24:46.745543 3088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:46.744241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38352 (local address 127.1.125.253:40729)
0810 07:24:46.744392 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:46.744396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:46.744576 (+ 180us) server_negotiation.cc:407] Connection header received
0810 07:24:46.744684 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:46.744687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:46.744738 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:46.744822 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:46.745435 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38352: BlockingRecv error: recv got EOF from 127.0.0.1:38352 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 07:24:47.005144 3088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:47.003899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38362 (local address 127.1.125.253:40729)
0810 07:24:47.004059 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:47.004063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:47.004202 (+ 139us) server_negotiation.cc:407] Connection header received
0810 07:24:47.004360 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:47.004362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:47.004416 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:47.004533 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:47.004995 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38362: BlockingRecv error: recv got EOF from 127.0.0.1:38362 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 07:24:47.316749 3251 catalog_manager.cc:1259] Loaded cluster ID: decbbffe0285411192bb852eef995c19
I20260810 07:24:47.316872 3251 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953: loading cluster ID for follower catalog manager: success
I20260810 07:24:47.318641 3251 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953: acquiring CA information for follower catalog manager: success
I20260810 07:24:47.319841 3251 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4932c716edfa42bc9702b00c1a46d953: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:24:47.333171 3291 catalog_manager.cc:1259] Loaded cluster ID: decbbffe0285411192bb852eef995c19
I20260810 07:24:47.333248 3291 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff: loading cluster ID for follower catalog manager: success
I20260810 07:24:47.334937 3291 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff: acquiring CA information for follower catalog manager: success
I20260810 07:24:47.336148 3291 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 725d16f713034ccfa081978a66a78dff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:24:47.523490 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:47.519651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38370 (local address 127.1.125.253:40729)
0810 07:24:47.519914 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:47.519918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:47.519993 (+ 75us) server_negotiation.cc:407] Connection header received
0810 07:24:47.520153 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:47.520156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:47.520291 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:47.520413 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:47.520736 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:47.521269 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:47.522107 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:47.522334 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:47.522417 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:47.522709 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:47.523157 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:47.523253 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:47.523310 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 07:24:47.523361 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":112,"threads_started":1}
I20260810 07:24:47.530754 3404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:47.526734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.1.125.252:41859)
0810 07:24:47.527020 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:47.527024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:47.527039 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:24:47.527227 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:47.527230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:47.527351 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:47.527476 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:47.527716 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:47.528304 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:47.529306 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:47.529535 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:47.529642 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:47.529952 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:47.530336 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:47.530476 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:47.530539 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 07:24:47.530595 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":99,"threads_started":1}
I20260810 07:24:47.538335 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:47.533955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38374 (local address 127.1.125.253:40729)
0810 07:24:47.534090 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:47.534096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:47.534609 (+ 513us) server_negotiation.cc:407] Connection header received
0810 07:24:47.534706 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:47.534710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:47.534790 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:47.534864 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:47.535253 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:47.535925 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:47.537201 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:47.537488 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:47.537608 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:47.537879 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:47.537955 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:47.538035 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:47.538130 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 07:24:47.538182 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 07:24:47.539196 3414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:47.533899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51558 (local address 127.1.125.254:43295)
0810 07:24:47.534707 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:47.534711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:47.534724 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:24:47.534815 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:47.534821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:47.534983 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:47.535111 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:47.535504 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:47.536328 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:47.537305 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:47.537710 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:47.537820 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:47.538307 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:47.538665 (+ 358us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:47.538771 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:47.538912 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 07:24:47.539013 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":79,"threads_started":1}
I20260810 07:24:47.539320 3404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:47.534255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42780 (local address 127.1.125.252:41859)
0810 07:24:47.534370 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:47.534374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:47.534735 (+ 361us) server_negotiation.cc:407] Connection header received
0810 07:24:47.534965 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:47.534969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:47.535021 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:47.535076 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:47.535531 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:47.536301 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:47.537614 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:47.537980 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:47.538077 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:47.538728 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:47.538813 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:47.538897 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:47.539114 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 07:24:47.539170 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:24:47.542882 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 2868
I20260810 07:24:47.555986 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 2981
I20260810 07:24:47.571168 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 3092
2026-08-10T07:24:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3336 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:24:47 dist-test-slave-44d2 krb5kdc[3421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:24:47 dist-test-slave-44d2 krb5kdc[3421](info): set up 2 sockets
Aug 10 07:24:47 dist-test-slave-44d2 krb5kdc[3421](info): commencing operation
krb5kdc: starting...
W20260810 07:24:48.848791 1527 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/test-user.keytab.
Aug 10 07:24:48 dist-test-slave-44d2 krb5kdc[3421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:24:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:24:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
Entry for principal kudu/127.1.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab.
I20260810 07:24:48.979152 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:43765
--webserver_interface=127.1.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:43765,127.1.125.253:42097,127.1.125.252:39291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:24:49.093520 3443 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:24:49.093858 3443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:49.093926 3443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:49.094009 3443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:49.098040 3443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:49.098126 3443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:49.098181 3443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:49.098237 3443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:49.098274 3443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:49.103605 3443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:43765,127.1.125.253:42097,127.1.125.252:39291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.254.keytab
--principal=kudu/127.1.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.254:43765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:49.104714 3443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:49.105697 3443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:49.114545 3458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:49.114534 3455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:49.114557 3460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:49.114689 3443 server_base.cc:1034] running on GCE node
I20260810 07:24:49.115483 3443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:49.116559 3443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:49.117729 3443 hybrid_clock.cc:648] HybridClock initialized: now 1786346689117708 us; error 32 us; skew 500 ppm
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:49.120664 3443 init.cc:377] Logged in from keytab as kudu/127.1.125.254@KRBTEST.COM (short username kudu)
I20260810 07:24:49.121598 3443 webserver.cc:460] Webserver started at http://127.1.125.254:41371/ using document root <none> and password file <none>
I20260810 07:24:49.122179 3443 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:49.122241 3443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:49.122411 3443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:49.124389 3443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/data/instance:
uuid: "65f9513887c94a748102fcd22d9cab03"
format_stamp: "Formatted at 2026-08-10 07:24:49 on dist-test-slave-44d2"
server_key: "1209c40240155bef83631b323c592dae"
server_key_iv: "a05e566b4c308031af80f84174227f71"
server_key_version: "encryptionkey@0"
I20260810 07:24:49.124984 3443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/wal/instance:
uuid: "65f9513887c94a748102fcd22d9cab03"
format_stamp: "Formatted at 2026-08-10 07:24:49 on dist-test-slave-44d2"
server_key: "1209c40240155bef83631b323c592dae"
server_key_iv: "a05e566b4c308031af80f84174227f71"
server_key_version: "encryptionkey@0"
I20260810 07:24:49.128757 3443 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:24:49.131086 3468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:49.132366 3443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:24:49.132563 3443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/wal
uuid: "65f9513887c94a748102fcd22d9cab03"
format_stamp: "Formatted at 2026-08-10 07:24:49 on dist-test-slave-44d2"
server_key: "1209c40240155bef83631b323c592dae"
server_key_iv: "a05e566b4c308031af80f84174227f71"
server_key_version: "encryptionkey@0"
I20260810 07:24:49.132688 3443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:49.152781 3443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:49.158934 3443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:49.159160 3443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:49.166425 3443 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.254:43765
I20260810 07:24:49.166757 3548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.254:43765 every 8 connection(s)
I20260810 07:24:49.167443 3443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/data/info.pb
I20260810 07:24:49.170378 3550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:49.174741 3550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:49.176544 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 3443
I20260810 07:24:49.176671 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-0/wal/instance
I20260810 07:24:49.176981 1527 external_mini_cluster.cc:1428] Setting key 3823ee286a3f71c5a949311816730784
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.253@KRBTEST.COM" created.
I20260810 07:24:49.191251 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.177081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:43765 (local address 127.0.0.1:35582)
0810 07:24:49.177578 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.177633 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:49.178522 (+ 889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:49.179344 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:49.179349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:49.179823 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:49.180081 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.180094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.181327 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.181333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.182578 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.182594 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.183305 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.183313 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.183489 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.184102 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:49.184124 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:49.185945 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:49.187989 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.188000 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.188011 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.188250 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.188640 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.188645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.188647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.188769 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.189421 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:49.189426 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:49.189613 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 07:24:49.189824 (+ 211us) client_negotiation.cc:240] Negotiation successful
0810 07:24:49.190234 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":115,"thread_start_us":133,"threads_started":1}
I20260810 07:24:49.191251 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.177081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35582 (local address 127.1.125.254:43765)
0810 07:24:49.178059 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.178073 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.178115 (+ 42us) server_negotiation.cc:407] Connection header received
0810 07:24:49.178834 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.178881 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.179158 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.179346 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.180237 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.181197 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.182745 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.183121 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.183255 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.186104 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.186118 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.186129 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.186151 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.187879 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.188366 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.188372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.188374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.188528 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.188885 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.188888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.188890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.189287 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.189452 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.189892 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.190154 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":863,"spinlock_wait_cycles":768,"thread_start_us":153,"threads_started":1}
I20260810 07:24:49.193856 3550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:49.194701 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.194378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:42097 (local address 127.0.0.1:42030)
0810 07:24:49.194520 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.194626 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:42097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 07:24:49.194896 3474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.125.253:42097: connect: Connection refused (error 111)
I20260810 07:24:49.195694 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.195477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:42097 (local address 127.0.0.1:42044)
0810 07:24:49.195588 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.195634 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:42097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 07:24:49.195868 3550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.253:42097: Network error: Client connection negotiation failed: client connection to 127.1.125.253:42097: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.253.keytab.
Entry for principal kudu/127.1.125.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.253.keytab.
I20260810 07:24:49.207458 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.253:42097
--webserver_interface=127.1.125.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.253.keytab
--principal=kudu/127.1.125.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:43765,127.1.125.253:42097,127.1.125.252:39291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:24:49.245208 3550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } attempt: 1
I20260810 07:24:49.246209 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.245832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:42097 (local address 127.0.0.1:42054)
0810 07:24:49.246026 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.246108 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:42097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 07:24:49.247058 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.246808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:42097 (local address 127.0.0.1:42068)
0810 07:24:49.246927 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.246985 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:42097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 07:24:49.247224 3550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.253:42097: Network error: Client connection negotiation failed: client connection to 127.1.125.253:42097: connect: Connection refused (error 111)
I20260810 07:24:49.315416 3550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } attempt: 2
I20260810 07:24:49.316299 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.315951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:42097 (local address 127.0.0.1:42084)
0810 07:24:49.316136 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.316209 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:42097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 07:24:49.317114 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.316860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:42097 (local address 127.0.0.1:42086)
0810 07:24:49.316982 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.317039 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.253:42097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 07:24:49.317260 3550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.253:42097: Network error: Client connection negotiation failed: client connection to 127.1.125.253:42097: connect: Connection refused (error 111)
W20260810 07:24:49.324265 3559 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:24:49.324579 3559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:49.324608 3559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:49.324678 3559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:49.328713 3559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:49.328763 3559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:49.328783 3559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:49.328835 3559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:49.328871 3559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:49.334175 3559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:43765,127.1.125.253:42097,127.1.125.252:39291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.253.keytab
--principal=kudu/127.1.125.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.253:42097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.125.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:49.335208 3559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:49.336216 3559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:49.344803 3571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:49.344859 3565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:49.344799 3568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:49.345158 3559 server_base.cc:1034] running on GCE node
I20260810 07:24:49.345789 3559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:49.346881 3559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:49.348085 3559 hybrid_clock.cc:648] HybridClock initialized: now 1786346689348069 us; error 28 us; skew 500 ppm
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:49.350968 3559 init.cc:377] Logged in from keytab as kudu/127.1.125.253@KRBTEST.COM (short username kudu)
I20260810 07:24:49.351871 3559 webserver.cc:460] Webserver started at http://127.1.125.253:37251/ using document root <none> and password file <none>
I20260810 07:24:49.352443 3559 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:49.352571 3559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:49.352830 3559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:49.354859 3559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/data/instance:
uuid: "99810b9cef6e46a58fe434971aadf3a5"
format_stamp: "Formatted at 2026-08-10 07:24:49 on dist-test-slave-44d2"
server_key: "07935377a942faaf68279299a7c996c0"
server_key_iv: "0b12f2f1b9e0f213af90ff89dd86e445"
server_key_version: "encryptionkey@0"
I20260810 07:24:49.355445 3559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/wal/instance:
uuid: "99810b9cef6e46a58fe434971aadf3a5"
format_stamp: "Formatted at 2026-08-10 07:24:49 on dist-test-slave-44d2"
server_key: "07935377a942faaf68279299a7c996c0"
server_key_iv: "0b12f2f1b9e0f213af90ff89dd86e445"
server_key_version: "encryptionkey@0"
I20260810 07:24:49.359239 3559 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260810 07:24:49.361706 3580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:49.362870 3559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:24:49.363046 3559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/wal
uuid: "99810b9cef6e46a58fe434971aadf3a5"
format_stamp: "Formatted at 2026-08-10 07:24:49 on dist-test-slave-44d2"
server_key: "07935377a942faaf68279299a7c996c0"
server_key_iv: "0b12f2f1b9e0f213af90ff89dd86e445"
server_key_version: "encryptionkey@0"
I20260810 07:24:49.363164 3559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:49.382277 3559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:49.385756 3559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:49.385977 3559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:49.393600 3559 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.253:42097
I20260810 07:24:49.393898 3667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.253:42097 every 8 connection(s)
I20260810 07:24:49.394678 3559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/data/info.pb
I20260810 07:24:49.397717 3668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:49.401605 3668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:49.404950 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 3559
I20260810 07:24:49.405115 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-1/wal/instance
I20260810 07:24:49.405419 1527 external_mini_cluster.cc:1428] Setting key 2db9795d8368d085420db8b38de3bcea
I20260810 07:24:49.408418 3550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } attempt: 3
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.253@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for kudu/127.1.125.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.125.252@KRBTEST.COM; defaulting to no policy
I20260810 07:24:49.417436 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.403563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35584 (local address 127.1.125.254:43765)
0810 07:24:49.403713 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.403716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.404616 (+ 900us) server_negotiation.cc:407] Connection header received
0810 07:24:49.405709 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.405713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.405779 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.405886 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.407075 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.407943 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.409485 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.409914 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.410045 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.412755 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.412761 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.412769 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.412791 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.414564 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.415075 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.415079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.415081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.415168 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.415512 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.415515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.415517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.416400 (+ 883us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.416552 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.417120 (+ 568us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.417268 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":60}
I20260810 07:24:49.418300 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.403900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:43765 (local address 127.0.0.1:35584)
0810 07:24:49.404381 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.404463 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:49.405483 (+ 1020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:49.406084 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:49.406092 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:49.406444 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:49.406924 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.406938 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.408094 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.408098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.409349 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.409361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.410102 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.410106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.410326 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.411047 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:49.411071 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:49.412615 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:49.414694 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.414699 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.414710 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.414954 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.415259 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.415263 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.415265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.415399 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.416535 (+ 1136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:49.416541 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:49.416830 (+ 289us) client_negotiation.cc:769] Sending connection context
0810 07:24:49.417056 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 07:24:49.417500 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":5,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.125.252@KRBTEST.COM" created.
I20260810 07:24:49.419934 3668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:49.420363 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.408910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:42097 (local address 127.0.0.1:42092)
0810 07:24:49.409120 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.409137 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:49.409217 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:49.409804 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:49.409807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:49.410036 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:49.410178 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.410186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.411810 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.411817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.412843 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.412859 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.413999 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.414004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.414141 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.414649 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:49.414663 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:49.416246 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:49.418665 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.418668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.418670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.418900 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.419253 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.419258 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.419262 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.419364 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.419860 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:49.419863 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:49.419972 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 07:24:49.420119 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 07:24:49.420251 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":3}
I20260810 07:24:49.420979 3674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.409013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42092 (local address 127.1.125.253:42097)
0810 07:24:49.409410 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.409431 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.409454 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:24:49.409499 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.409509 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.409654 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.409815 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.410315 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.411661 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.413319 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.413874 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.413998 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.416397 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.416424 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.416436 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.416457 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.418518 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.419018 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.419027 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.419032 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.419129 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.419470 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.419474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.419475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.419756 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.419891 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.420652 (+ 761us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.420800 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":307,"thread_start_us":133,"threads_started":1}
I20260810 07:24:49.422940 3550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:49.423624 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.423380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57304)
0810 07:24:49.423498 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.423553 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 07:24:49.424288 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.424049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57306)
0810 07:24:49.424188 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.424232 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260810 07:24:49.424508 3550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:39291: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.253@KRBTEST.COM for kudu/127.1.125.253@KRBTEST.COM
I20260810 07:24:49.430320 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.420460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:42097 (local address 127.0.0.1:42106)
0810 07:24:49.420905 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.420921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:49.421011 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:49.421236 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:49.421238 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:49.421392 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:49.421467 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.421472 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.422245 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.422250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.423172 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.423188 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.424089 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.424095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.424205 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.424750 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:49.424771 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:49.426077 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:49.428908 (+ 2831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.428911 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.428912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.429129 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.429438 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.429440 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.429441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.429537 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.429976 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:49.429977 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:49.430052 (+ 75us) client_negotiation.cc:769] Sending connection context
0810 07:24:49.430130 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:24:49.430221 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366}
I20260810 07:24:49.430651 3676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.420734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42106 (local address 127.1.125.253:42097)
0810 07:24:49.421032 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.421043 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.421057 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:24:49.421089 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.421092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.421125 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.421227 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.421573 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.422118 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.423315 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.423823 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.423964 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.426240 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.426247 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.426251 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.426284 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.428797 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.429234 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.429237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.429239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.429337 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.429637 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.429641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.429643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.429876 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.429990 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.430293 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.430391 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":115,"threads_started":1}
I20260810 07:24:49.431308 3668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:49.432072 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.431779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57316)
0810 07:24:49.431918 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.432009 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 07:24:49.432298 3582 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
I20260810 07:24:49.433137 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.432950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57318)
0810 07:24:49.433052 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.433092 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 07:24:49.433326 3668 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:39291: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.125.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.252.keytab.
Entry for principal kudu/127.1.125.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.252.keytab.
I20260810 07:24:49.439109 1527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
/tmp/dist-test-taskwKbuEn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.252:39291
--webserver_interface=127.1.125.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.252.keytab
--principal=kudu/127.1.125.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.125.212:39695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.125.254:43765,127.1.125.253:42097,127.1.125.252:39291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:24:49.455663 3550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } attempt: 1
I20260810 07:24:49.456526 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.456183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57326)
0810 07:24:49.456331 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.456398 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 07:24:49.457285 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.457047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57330)
0810 07:24:49.457170 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.457219 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 07:24:49.457443 3550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:39291: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
I20260810 07:24:49.482662 3668 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } attempt: 1
I20260810 07:24:49.483776 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.483322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57332)
0810 07:24:49.483567 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.483664 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 07:24:49.484794 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.484534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57342)
0810 07:24:49.484668 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.484726 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 07:24:49.484939 3668 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:39291: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
I20260810 07:24:49.532637 3550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } attempt: 2
I20260810 07:24:49.533519 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.533212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57346)
0810 07:24:49.533368 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.533439 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 07:24:49.534268 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.534050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57362)
0810 07:24:49.534164 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.534211 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 07:24:49.534488 3550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:39291: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
I20260810 07:24:49.553123 3668 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } attempt: 2
I20260810 07:24:49.554023 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.553656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57370)
0810 07:24:49.553823 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.553894 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 07:24:49.554774 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.554552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57376)
0810 07:24:49.554670 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.554719 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 07:24:49.554900 3668 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.125.252:39291: Network error: Client connection negotiation failed: client connection to 127.1.125.252:39291: connect: Connection refused (error 111)
W20260810 07:24:49.555667 3680 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:24:49.555931 3680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:24:49.555956 3680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:24:49.555994 3680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:24:49.560012 3680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:24:49.560070 3680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:24:49.560088 3680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:24:49.560105 3680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:24:49.560120 3680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:24:49.565429 3680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.125.212:39695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.125.254:43765,127.1.125.253:42097,127.1.125.252:39291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/krb5kdc/kudu_127.1.125.252.keytab
--principal=kudu/127.1.125.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.125.252:39291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.125.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:24:49.566524 3680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:24:49.567667 3680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:24:49.576455 3692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:49.576448 3688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:24:49.576510 3686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:24:49.576710 3680 server_base.cc:1034] running on GCE node
I20260810 07:24:49.577436 3680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:24:49.578876 3680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:24:49.580073 3680 hybrid_clock.cc:648] HybridClock initialized: now 1786346689580041 us; error 44 us; skew 500 ppm
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:24:49.582957 3680 init.cc:377] Logged in from keytab as kudu/127.1.125.252@KRBTEST.COM (short username kudu)
I20260810 07:24:49.583858 3680 webserver.cc:460] Webserver started at http://127.1.125.252:45255/ using document root <none> and password file <none>
I20260810 07:24:49.584398 3680 fs_manager.cc:359] Metadata directory not provided
I20260810 07:24:49.584456 3680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:24:49.584704 3680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:24:49.586789 3680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/data/instance:
uuid: "af4a1a2110e6451e8b17291884552716"
format_stamp: "Formatted at 2026-08-10 07:24:49 on dist-test-slave-44d2"
server_key: "a8a1fbbb9dd040fda6b3e71c97ee9b65"
server_key_iv: "a5b907934265d1c5b724ac2fa75f3786"
server_key_version: "encryptionkey@0"
I20260810 07:24:49.587329 3680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/wal/instance:
uuid: "af4a1a2110e6451e8b17291884552716"
format_stamp: "Formatted at 2026-08-10 07:24:49 on dist-test-slave-44d2"
server_key: "a8a1fbbb9dd040fda6b3e71c97ee9b65"
server_key_iv: "a5b907934265d1c5b724ac2fa75f3786"
server_key_version: "encryptionkey@0"
I20260810 07:24:49.591251 3680 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:24:49.593760 3702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:24:49.595029 3680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:24:49.595192 3680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/data,/tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/wal
uuid: "af4a1a2110e6451e8b17291884552716"
format_stamp: "Formatted at 2026-08-10 07:24:49 on dist-test-slave-44d2"
server_key: "a8a1fbbb9dd040fda6b3e71c97ee9b65"
server_key_iv: "a5b907934265d1c5b724ac2fa75f3786"
server_key_version: "encryptionkey@0"
I20260810 07:24:49.595309 3680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:24:49.612887 3680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:24:49.615912 3680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:24:49.616138 3680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:24:49.624356 3680 rpc_server.cc:307] RPC server started. Bound to: 127.1.125.252:39291
I20260810 07:24:49.624672 3791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.125.252:39291 every 8 connection(s)
I20260810 07:24:49.625428 3680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/data/info.pb
I20260810 07:24:49.625876 1527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu as pid 3680
I20260810 07:24:49.626024 1527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwKbuEn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346386457763-1527-0/minicluster-data/master-2/wal/instance
I20260810 07:24:49.626339 1527 external_mini_cluster.cc:1428] Setting key 828bd191b7fa6ad78c99cd36bdc4b14f
I20260810 07:24:49.628644 3792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:24:49.632699 3792 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:49.633667 3550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } attempt: 3
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.252@KRBTEST.COM
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.254@KRBTEST.COM for kudu/127.1.125.252@KRBTEST.COM
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.252@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:24:49.643160 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.627684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.1.125.252:39291)
0810 07:24:49.628101 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.628130 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.628370 (+ 240us) server_negotiation.cc:407] Connection header received
0810 07:24:49.629229 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.629247 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.629598 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.629938 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.630772 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.631776 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.632728 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.633267 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.633440 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.636935 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.636949 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.636958 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.636986 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.639023 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.639534 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.639540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.639546 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.639727 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.640059 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.640063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.640065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.640695 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.640890 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.641737 (+ 847us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.642105 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":145,"threads_started":1}
I20260810 07:24:49.645810 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.635369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35600 (local address 127.1.125.254:43765)
0810 07:24:49.635528 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.635533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.635549 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:24:49.635583 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.635585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.635618 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.635710 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.636662 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.637857 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.639187 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.639568 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.639681 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.642271 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.642275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.642278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.642302 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.643932 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.644376 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.644379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.644381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.644472 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.644852 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.644855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.644857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.645060 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.645168 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.645560 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.645684 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":576,"server-negotiator.queue_time_us":57}
I20260810 07:24:49.645836 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.634500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.254:43765 (local address 127.0.0.1:35600)
0810 07:24:49.634906 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.634961 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:49.635069 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:49.635764 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:49.635772 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:49.636208 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:49.636525 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.636537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.637995 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.637998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.639068 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.639080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.639690 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.639694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.639828 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.640328 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:49.640341 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:49.642126 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:49.644028 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.644037 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.644041 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.644275 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.644595 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.644598 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.644600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.644726 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.645169 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:49.645176 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:49.645302 (+ 126us) client_negotiation.cc:769] Sending connection context
0810 07:24:49.645460 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 07:24:49.645648 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":89,"thread_start_us":110,"threads_started":1}
I20260810 07:24:49.646267 3668 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } attempt: 3
I20260810 07:24:49.646885 3792 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:49.647109 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.634153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57390)
0810 07:24:49.634319 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.634339 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:49.634443 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:49.634835 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:49.634838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:49.635050 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:49.635197 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.635204 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.636540 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.636544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.637227 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.637245 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.638436 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.638439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.638521 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.638965 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:49.638980 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:49.640545 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:49.643364 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.643368 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.643371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.643702 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.644774 (+ 1072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.644777 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.644779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.644868 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.645295 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:49.645298 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:49.646700 (+ 1402us) client_negotiation.cc:769] Sending connection context
0810 07:24:49.646799 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 07:24:49.646929 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":3}
I20260810 07:24:49.648021 3797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.634217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57390 (local address 127.1.125.252:39291)
0810 07:24:49.634562 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.634573 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.634583 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:24:49.634613 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.634615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.634678 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.635320 (+ 642us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.635443 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.636357 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.637996 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.638311 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.638437 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.640708 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.640713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.640716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.640743 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.643215 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.644570 (+ 1355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.644574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.644579 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.644680 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.644971 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.644974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.644976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.645197 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.645300 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.647709 (+ 2409us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.647827 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":273,"thread_start_us":135,"threads_started":1}
I20260810 07:24:49.651707 3550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03: Bootstrap starting.
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.253@KRBTEST.COM for kudu/127.1.125.252@KRBTEST.COM
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.254@KRBTEST.COM
I20260810 07:24:49.655086 3550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03: Neither blocks nor log segments found. Creating new log.
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.252@KRBTEST.COM for kudu/127.1.125.253@KRBTEST.COM
I20260810 07:24:49.656169 3550 log.cc:824] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:49.656929 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.646652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35608 (local address 127.1.125.254:43765)
0810 07:24:49.646753 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.646758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.646771 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:24:49.646805 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.646808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.646842 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.646908 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.647722 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.648397 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.649382 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.649657 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.649764 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.653383 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.653387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.653390 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.653420 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.654812 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.655333 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.655336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.655338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.655444 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.655782 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.655785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.655787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.656044 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.656194 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.656649 (+ 455us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.656783 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:24:49.657034 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.646770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57396)
0810 07:24:49.647330 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.647347 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:49.647439 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:49.647836 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:49.647839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:49.648016 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:49.648166 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.648176 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.649184 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.649188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.650002 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.650012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.650581 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.650585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.650667 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.651106 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:49.651119 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:49.652636 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:49.654948 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.654954 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.654959 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.655254 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.655673 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.655676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.655678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.655776 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.656389 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:49.656392 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:49.656561 (+ 169us) client_negotiation.cc:769] Sending connection context
0810 07:24:49.656658 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 07:24:49.656830 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"spinlock_wait_cycles":8960}
I20260810 07:24:49.658195 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.646998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57396 (local address 127.1.125.252:39291)
0810 07:24:49.647406 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.647409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.647425 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:24:49.647669 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.647672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.647718 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.647813 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.648306 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.649042 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.650135 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.650455 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.650866 (+ 411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.652757 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.652761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.652763 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.652788 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.654807 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.655463 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.655467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.655468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.655556 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.655894 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.655896 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.655898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.656196 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.657826 (+ 1630us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.657906 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.658037 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340}
I20260810 07:24:49.659245 3550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03: No bootstrap required, opened a new log
I20260810 07:24:49.660054 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.647322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.253:42097 (local address 127.0.0.1:42122)
0810 07:24:49.649443 (+ 2121us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.649456 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:49.649583 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:49.649875 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:49.649878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:49.650047 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:49.650166 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.650172 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.651107 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.651110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.651930 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.651942 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.652785 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.652789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.652908 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.653704 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:49.653724 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:49.656260 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:49.658302 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.658308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.658313 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.658552 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.658918 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.658923 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.658928 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.659031 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.659576 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:49.659582 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:49.659679 (+ 97us) client_negotiation.cc:769] Sending connection context
0810 07:24:49.659777 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 07:24:49.659901 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2049}
I20260810 07:24:49.660635 3676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.647547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42122 (local address 127.1.125.253:42097)
0810 07:24:49.647681 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.647685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.649520 (+ 1835us) server_negotiation.cc:407] Connection header received
0810 07:24:49.649678 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.649681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.649721 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.649766 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.650281 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.650972 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.652069 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.652400 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.652532 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.656409 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.656413 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.656416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.656440 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.658175 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.658705 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.658708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.658710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.658806 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.659143 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.659146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.659148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.659429 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.659524 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.660349 (+ 825us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.660466 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 07:24:49.662099 3668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5: Bootstrap starting.
I20260810 07:24:49.662992 3550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } }
I20260810 07:24:49.663216 3550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65f9513887c94a748102fcd22d9cab03, State: Initialized, Role: FOLLOWER
I20260810 07:24:49.663559 3550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } }
I20260810 07:24:49.663803 3792 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:24:49.664141 3800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } } }
I20260810 07:24:49.664335 3800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:49.665175 3550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [sys.catalog]: configured and running, proceeding with master startup.
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.125.253@KRBTEST.COM
I20260810 07:24:49.665866 3668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:49.667320 3668 log.cc:824] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:49.669679 3668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5: No bootstrap required, opened a new log
Aug 10 07:24:49 dist-test-slave-44d2 krb5kdc[3421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.125.252@KRBTEST.COM for kudu/127.1.125.252@KRBTEST.COM
W20260810 07:24:49.672837 3818 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:49.673311 3818 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:49.673789 3674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.658861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42136 (local address 127.1.125.253:42097)
0810 07:24:49.658975 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.658980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.658993 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:24:49.659025 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.659029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.659068 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.659157 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.660057 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.660829 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.662363 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.662688 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.662799 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.666200 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.666205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.666208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.666240 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.668069 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.671359 (+ 3290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.671362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.671364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.671461 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.671767 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.671773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.671776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.671992 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.672449 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.673160 (+ 711us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.673615 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 07:24:49.674062 3668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } }
I20260810 07:24:49.674296 3668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99810b9cef6e46a58fe434971aadf3a5, State: Initialized, Role: FOLLOWER
I20260810 07:24:49.674677 3668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } }
I20260810 07:24:49.671563 3443 master_runner.cc:186] Error getting master registration for 127.1.125.253:42097: OK, instance_id { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" instance_seqno: 1786346689391369 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:24:49.675218 3823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } } }
I20260810 07:24:49.675349 3443 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:24:49.675366 3823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:49.676113 3668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:49.676396 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.664350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.125.252:39291 (local address 127.0.0.1:57398)
0810 07:24:49.664644 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 07:24:49.664658 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:24:49.664734 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:24:49.666890 (+ 2156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:24:49.666893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:24:49.667071 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:24:49.667450 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.667459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.668370 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.668374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.669145 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:24:49.669159 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.669668 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.669672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:24:49.669742 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.670227 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:24:49.670242 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:24:49.671768 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:24:49.674435 (+ 2667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.674438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.674440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.674668 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.675247 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:24:49.675250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:24:49.675252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:24:49.675325 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:24:49.675846 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:24:49.675850 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:24:49.675972 (+ 122us) client_negotiation.cc:769] Sending connection context
0810 07:24:49.676106 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 07:24:49.676242 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260810 07:24:49.677018 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.664602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.1.125.252:39291)
0810 07:24:49.666672 (+ 2070us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.666678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.666692 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:24:49.666734 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.666737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.666781 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.666878 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.667574 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.668252 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.669272 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.669544 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.670859 (+ 1315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.671904 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.671907 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.671910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.671932 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.674209 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.675061 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.675064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.675066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.675151 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.675514 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.675517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.675518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.675716 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.675820 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.676752 (+ 932us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.676863 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1972}
I20260810 07:24:49.679368 1527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:24:49.680616 3792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716: Bootstrap starting.
I20260810 07:24:49.680644 3512 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "65f9513887c94a748102fcd22d9cab03"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35608
I20260810 07:24:49.680866 3512 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 07:24:49.680979 3512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:49.682396 3512 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } }
I20260810 07:24:49.683387 3512 leader_election.cc:288] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [CANDIDATE]: Term 1 election: Requested vote from peers 99810b9cef6e46a58fe434971aadf3a5 (127.1.125.253:42097), af4a1a2110e6451e8b17291884552716 (127.1.125.252:39291)
I20260810 07:24:49.683570 3792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716: Neither blocks nor log segments found. Creating new log.
I20260810 07:24:49.684168 3626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65f9513887c94a748102fcd22d9cab03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "99810b9cef6e46a58fe434971aadf3a5"
I20260810 07:24:49.684420 3626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:49.684567 3792 log.cc:824] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716: Log is configured to *not* fsync() on all Append() calls
I20260810 07:24:49.686005 3626 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65f9513887c94a748102fcd22d9cab03 in term 1.
I20260810 07:24:49.686451 3474 leader_election.cc:302] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65f9513887c94a748102fcd22d9cab03, 99810b9cef6e46a58fe434971aadf3a5; no voters:
I20260810 07:24:49.686754 3800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:24:49.687009 3800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [term 1 LEADER]: Becoming Leader. State: Replica: 65f9513887c94a748102fcd22d9cab03, State: Running, Role: LEADER
I20260810 07:24:49.687345 3800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } }
I20260810 07:24:49.687434 3792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716: No bootstrap required, opened a new log
I20260810 07:24:49.689257 3800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65f9513887c94a748102fcd22d9cab03. Latest consensus state: current_term: 1 leader_uuid: "65f9513887c94a748102fcd22d9cab03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } } }
I20260810 07:24:49.689397 3800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:49.692044 3792 raft_consensus.cc:348] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } }
I20260810 07:24:49.692526 3792 raft_consensus.cc:729] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af4a1a2110e6451e8b17291884552716, State: Initialized, Role: FOLLOWER
I20260810 07:24:49.693120 3792 consensus_queue.cc:260] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } }
I20260810 07:24:49.693727 3859 sys_catalog.cc:455] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } } }
I20260810 07:24:49.693986 3859 sys_catalog.cc:458] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:49.694507 3792 sys_catalog.cc:564] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:24:49.694586 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65f9513887c94a748102fcd22d9cab03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "af4a1a2110e6451e8b17291884552716"
I20260810 07:24:49.694777 3743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:24:49.696023 3743 raft_consensus.cc:2454] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65f9513887c94a748102fcd22d9cab03 in term 1.
I20260810 07:24:49.699975 3856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:24:49.694111 3559 master_runner.cc:186] Error getting master registration for 127.1.125.252:39291: OK, instance_id { permanent_uuid: "af4a1a2110e6451e8b17291884552716" instance_seqno: 1786346689622397 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:24:49.703994 3559 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:24:49.704811 3841 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:49.704891 3841 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:49.705205 3856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260810 07:24:49.706349 3879 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:24:49.706442 3879 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:24:49.714993 3674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.697605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42146 (local address 127.1.125.253:42097)
0810 07:24:49.697707 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.697712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.705200 (+ 7488us) server_negotiation.cc:407] Connection header received
0810 07:24:49.705376 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.705379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.705429 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.705482 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.706500 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.707300 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.708403 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.708747 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.708826 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.711030 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.711034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.711036 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.711064 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.712977 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.713545 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.713549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.713550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.713641 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.713957 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.713961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.713962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.714200 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.714320 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.714710 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.714833 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:24:49.720829 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.698272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.1.125.252:39291)
0810 07:24:49.698377 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.698381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.705482 (+ 7101us) server_negotiation.cc:407] Connection header received
0810 07:24:49.705645 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.705648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.705698 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.705779 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.707368 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.708137 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.709557 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.709920 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.710015 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.716620 (+ 6605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.716624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.716627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.716653 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.718538 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.719330 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.719336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.719340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.719439 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.719784 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.719790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.719794 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.720026 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.720150 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.720532 (+ 382us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.720659 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:24:49.722062 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:49.696923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.1.125.254:43765)
0810 07:24:49.697038 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.697043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.698095 (+ 1052us) server_negotiation.cc:407] Connection header received
0810 07:24:49.698580 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.698583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.698648 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.698703 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.699906 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.700686 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.701824 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:49.702132 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:49.702210 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:49.717540 (+ 15330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:24:49.717548 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:24:49.717553 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:24:49.717597 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:24:49.720003 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.720519 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.720524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.720525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.720626 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:24:49.721036 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:24:49.721040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:24:49.721041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:24:49.721278 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:24:49.721372 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:49.721767 (+ 395us) server_negotiation.cc:299] Negotiation successful
0810 07:24:49.721898 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:24:49.722500 3743 raft_consensus.cc:1261] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716 [term 1 FOLLOWER]: Refusing update from remote peer 65f9513887c94a748102fcd22d9cab03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:49.723213 3626 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5 [term 1 FOLLOWER]: Refusing update from remote peer 65f9513887c94a748102fcd22d9cab03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:24:49.723794 3800 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:49.724228 3800 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:24:49.728569 3823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65f9513887c94a748102fcd22d9cab03. Latest consensus state: current_term: 1 leader_uuid: "65f9513887c94a748102fcd22d9cab03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } } }
I20260810 07:24:49.728683 3823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:49.732517 3859 sys_catalog.cc:455] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65f9513887c94a748102fcd22d9cab03. Latest consensus state: current_term: 1 leader_uuid: "65f9513887c94a748102fcd22d9cab03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } } }
I20260810 07:24:49.732678 3859 sys_catalog.cc:458] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:49.736454 3844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65f9513887c94a748102fcd22d9cab03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } } }
I20260810 07:24:49.736610 3844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:49.737443 3800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65f9513887c94a748102fcd22d9cab03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } } }
I20260810 07:24:49.737566 3800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03 [sys.catalog]: This master's current role is: LEADER
I20260810 07:24:49.738636 3859 sys_catalog.cc:455] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65f9513887c94a748102fcd22d9cab03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } } }
I20260810 07:24:49.738749 3859 sys_catalog.cc:458] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:49.739742 3823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65f9513887c94a748102fcd22d9cab03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f9513887c94a748102fcd22d9cab03" member_type: VOTER last_known_addr { host: "127.1.125.254" port: 43765 } } peers { permanent_uuid: "99810b9cef6e46a58fe434971aadf3a5" member_type: VOTER last_known_addr { host: "127.1.125.253" port: 42097 } } peers { permanent_uuid: "af4a1a2110e6451e8b17291884552716" member_type: VOTER last_known_addr { host: "127.1.125.252" port: 39291 } } }
I20260810 07:24:49.739856 3823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:24:49.741847 3887 mvcc.cc:205] Tried to move back new op lower bound from 7316876041081192448 to 7316876040963092480. Current Snapshot: MvccSnapshot[applied={T|T < 7316876041081192448}]
I20260810 07:24:49.742018 3886 mvcc.cc:205] Tried to move back new op lower bound from 7316876041081192448 to 7316876040963092480. Current Snapshot: MvccSnapshot[applied={T|T < 7316876041081192448}]
I20260810 07:24:49.742163 3889 mvcc.cc:205] Tried to move back new op lower bound from 7316876041081192448 to 7316876040963092480. Current Snapshot: MvccSnapshot[applied={T|T < 7316876041081192448}]
I20260810 07:24:49.747246 3856 catalog_manager.cc:1347] Generated new cluster ID: 1788cdce2d404b6a874f5b23bb2cd34f
I20260810 07:24:49.747313 3856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:24:49.762393 3856 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:24:49.763643 3856 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:24:49.780821 3856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65f9513887c94a748102fcd22d9cab03: Generated new TSK 0
I20260810 07:24:49.781698 3856 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:24:49.814935 3554 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:49.811568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35632 (local address 127.1.125.254:43765)
0810 07:24:49.811705 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.811709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.813438 (+ 1729us) server_negotiation.cc:407] Connection header received
0810 07:24:49.813888 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.813894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.813954 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.814016 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.814791 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35632: BlockingRecv error: recv got EOF from 127.0.0.1:35632 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260810 07:24:49.816123 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:49.812340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42148 (local address 127.1.125.253:42097)
0810 07:24:49.812455 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.812459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.813800 (+ 1341us) server_negotiation.cc:407] Connection header received
0810 07:24:49.815322 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.815326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.815381 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.815434 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.815996 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42148: BlockingRecv error: recv got EOF from 127.0.0.1:42148 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 07:24:49.817776 3796 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:49.813146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57416 (local address 127.1.125.252:39291)
0810 07:24:49.813269 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.813273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.816646 (+ 3373us) server_negotiation.cc:407] Connection header received
0810 07:24:49.817019 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.817024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.817079 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.817132 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.817653 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57416: BlockingRecv error: recv got EOF from 127.0.0.1:57416 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 07:24:49.822501 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:49.820931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42158 (local address 127.1.125.253:42097)
0810 07:24:49.821037 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.821041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.821479 (+ 438us) server_negotiation.cc:407] Connection header received
0810 07:24:49.821717 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.821721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.821798 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.821849 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.822378 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42158: BlockingRecv error: recv got EOF from 127.0.0.1:42158 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 07:24:49.827638 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:49.826088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42170 (local address 127.1.125.253:42097)
0810 07:24:49.826188 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.826192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.826744 (+ 552us) server_negotiation.cc:407] Connection header received
0810 07:24:49.826929 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.826932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.826980 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.827030 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.827537 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42170: BlockingRecv error: recv got EOF from 127.0.0.1:42170 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 07:24:49.833235 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:49.831616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42180 (local address 127.1.125.253:42097)
0810 07:24:49.831720 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.831724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.832263 (+ 539us) server_negotiation.cc:407] Connection header received
0810 07:24:49.832462 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.832466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.832552 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.832622 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.833145 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42180: BlockingRecv error: recv got EOF from 127.0.0.1:42180 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 07:24:49.840935 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:49.839224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42192 (local address 127.1.125.253:42097)
0810 07:24:49.839363 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.839368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.839954 (+ 586us) server_negotiation.cc:407] Connection header received
0810 07:24:49.840166 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.840169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.840220 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.840273 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.840833 (+ 560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42192: BlockingRecv error: recv got EOF from 127.0.0.1:42192 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260810 07:24:49.852424 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:49.850916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42206 (local address 127.1.125.253:42097)
0810 07:24:49.851020 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.851023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.851544 (+ 521us) server_negotiation.cc:407] Connection header received
0810 07:24:49.851726 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.851729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.851779 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.851827 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.852320 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42206: BlockingRecv error: recv got EOF from 127.0.0.1:42206 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 07:24:49.874886 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:49.873002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42208 (local address 127.1.125.253:42097)
0810 07:24:49.873128 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.873132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.873760 (+ 628us) server_negotiation.cc:407] Connection header received
0810 07:24:49.873982 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.873986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.874047 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.874101 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.874765 (+ 664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42208: BlockingRecv error: recv got EOF from 127.0.0.1:42208 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 07:24:49.910917 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:49.909484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42224 (local address 127.1.125.253:42097)
0810 07:24:49.909670 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.909676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.909939 (+ 263us) server_negotiation.cc:407] Connection header received
0810 07:24:49.910129 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.910132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.910185 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.910280 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.910782 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42224: BlockingRecv error: recv got EOF from 127.0.0.1:42224 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
W20260810 07:24:49.979207 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:49.978068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42228 (local address 127.1.125.253:42097)
0810 07:24:49.978205 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:49.978208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:49.978401 (+ 193us) server_negotiation.cc:407] Connection header received
0810 07:24:49.978536 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:49.978538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:49.978589 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:49.978672 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:49.979097 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42228: BlockingRecv error: recv got EOF from 127.0.0.1:42228 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 07:24:50.110862 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:50.109647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42240 (local address 127.1.125.253:42097)
0810 07:24:50.109812 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:50.109819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:50.109950 (+ 131us) server_negotiation.cc:407] Connection header received
0810 07:24:50.110080 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:50.110083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:50.110137 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:50.110211 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:50.110697 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42240: BlockingRecv error: recv got EOF from 127.0.0.1:42240 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 07:24:50.370210 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:24:50.369064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58674 (local address 127.1.125.253:42097)
0810 07:24:50.369236 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:50.369241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:50.369356 (+ 115us) server_negotiation.cc:407] Connection header received
0810 07:24:50.369476 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:50.369478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:50.369544 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:50.369620 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:24:50.370067 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58674: BlockingRecv error: recv got EOF from 127.0.0.1:58674 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 07:24:50.706301 3841 catalog_manager.cc:1259] Loaded cluster ID: 1788cdce2d404b6a874f5b23bb2cd34f
I20260810 07:24:50.706399 3841 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5: loading cluster ID for follower catalog manager: success
I20260810 07:24:50.707757 3879 catalog_manager.cc:1259] Loaded cluster ID: 1788cdce2d404b6a874f5b23bb2cd34f
I20260810 07:24:50.707834 3879 catalog_manager.cc:1552] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716: loading cluster ID for follower catalog manager: success
I20260810 07:24:50.708823 3841 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5: acquiring CA information for follower catalog manager: success
I20260810 07:24:50.709870 3879 catalog_manager.cc:1574] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716: acquiring CA information for follower catalog manager: success
I20260810 07:24:50.710310 3841 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 99810b9cef6e46a58fe434971aadf3a5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:24:50.711012 3879 catalog_manager.cc:1602] T 00000000000000000000000000000000 P af4a1a2110e6451e8b17291884552716: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:24:50.888574 4002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:50.884671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58688 (local address 127.1.125.253:42097)
0810 07:24:50.884945 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:50.884949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:50.884965 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:24:50.885083 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:50.885098 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:50.885233 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:50.885333 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:50.885602 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:50.886139 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:50.886966 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:50.887196 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:50.887273 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:50.887559 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:50.888011 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:50.888128 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:50.888281 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 07:24:50.888333 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":82,"threads_started":1}
I20260810 07:24:50.896874 4008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:50.892042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43636 (local address 127.1.125.252:39291)
0810 07:24:50.892311 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:50.892315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:50.892330 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:24:50.892506 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:50.892509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:50.892650 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:50.892753 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:50.893110 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:50.893841 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:50.894903 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:50.895200 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:50.895362 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:50.895772 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:50.896247 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:50.896423 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:50.896598 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 07:24:50.896669 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":114,"threads_started":1}
I20260810 07:24:50.904162 4008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:50.900209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43650 (local address 127.1.125.252:39291)
0810 07:24:50.900602 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:50.900611 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:50.900697 (+ 86us) server_negotiation.cc:407] Connection header received
0810 07:24:50.900797 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:50.900804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:50.900866 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:50.900939 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:50.901420 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:50.901977 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:50.902894 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:50.903189 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:50.903316 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:50.903670 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:50.903749 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:50.903791 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:50.903947 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 07:24:50.904004 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:24:50.904381 4002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:50.900364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58690 (local address 127.1.125.253:42097)
0810 07:24:50.900526 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:50.900530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:50.900867 (+ 337us) server_negotiation.cc:407] Connection header received
0810 07:24:50.900968 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:50.900971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:50.901040 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:50.901119 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:50.901499 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:50.901997 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:50.903257 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:50.903501 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:50.903619 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:50.903884 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:50.903960 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:50.904043 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:50.904159 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 07:24:50.904214 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:24:50.905905 4020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:24:50.900079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45182 (local address 127.1.125.254:43765)
0810 07:24:50.900634 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0810 07:24:50.900641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:24:50.900657 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:24:50.900763 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:24:50.900770 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:24:50.900944 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:24:50.901068 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:24:50.902162 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:50.902989 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:50.904114 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:24:50.904429 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:24:50.904609 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:24:50.905122 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:24:50.905479 (+ 357us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:24:50.905609 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 07:24:50.905685 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 07:24:50.905748 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":82,"threads_started":1}
I20260810 07:24:50.909001 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 3443
I20260810 07:24:50.924861 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 3559
I20260810 07:24:50.938771 1527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwKbuEn/build/debug/bin/kudu with pid 3680
2026-08-10T07:24:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3365 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6702 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (304471 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST